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PREFACE 



This report and the publication of a related monograph, Lessons on 
Transitional Employment , mark the culmination of the STETS demonstration. 
STETS was a major experiment testing the feasibility, cost and effective- 
ness of a transitional employment program for mentally retarded young 
adults. Starting in 1981, the Manpower Demonstration Research Corporation 
(MDRC), the nonprofit corporation that managed the demonstration, traced 
the experiences of five diverse organizations that aimed to prepare mental- 
ly retarded youths between the ages of 18 and 24 for competitive 
employment. The program drew on MDRC's National Supported Work Demonstra- 
tion — a structured work experience program shown to be successful in 
helping long-term welfare recipients obtain regular employment. Like 
Supported Work, STETS tried to acclimate participants to the regular work 
environment in gradual stages over about a year's time. 

In a 1984 implementation report, MDRC concluded that although J:he 
challenge was one of considerable magnitude, the program could be feasibly 
operated by a variety of agencies and could help many mentally retarded 
citizens make the transition to competitive jobs. At the same time, the 
study stressed that not all participants could be placed in competitive 
jobs and that alternative programs and services would still be needed. 

At that point, however, the most important question ~ whether STETS 
was actually effective - was still unanswered. To learn whether or not 
participants would have fared just as well without this intervention, it 
was necessary to compare outcomes for them and a matched control group, 
who, although they could be included in other programs, were not offered 
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STETS servicee* 

Such a comparison falls within the purview of the STETS impact 
analysis, which MDRC subcontracted to Mathematica Policy Research, Inc., 
and which is the subject of this report. This report presents findings for 
a group of over 400 experimentals and matched control group members at 6, 
15 and 22 months after enrollment. It also contains a benefit-cost 
analysis. 

The study offers convincing evidence that STETS is a promising 
approach. Although it is clear that transitional employment should be only 
one option within the mix of services available to the mentally regarded 
population, its importance can ts gauged by the reporf^s findings: About 
one year after leaving the program, participants were substantially more 
likely than their control group counterparts to be working in competitive 
jobs and less likely to be in sheltered workshops. The program also seemed 
to be particularly effective for some groups who may have special 
difficulty finding jobs on their own — for example, those in the 
moderately retarded category. 

Overall, STETS appears to be an effective investment of public 
resources. Economic benefits to society seem likely to exceed program 
costs within three years of enrollment. These findings, along with others 
from this study, should therefore hold considerable interest for those who 
have long sought more knowledge on whether mentally retarded citizens can 
be helped to move into the regular labor market. 



Barbara B, Blum 
President 
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EXECUTIVE SUMMARY 



The Structured Training and Emplo3,rcnt Transitional Services 
(STETS) demonstration was designed to provide the first rigorous test of 
the effectiveness of transitional-employment programs in integrating 
tjaentally retarded young adults into the economic and social mainstream. 
Under the demonstration, which was funded by the Employment and Training 
Administration of the U.S. Department of Labor and directed by the Manpower 
Demonstration Research Corporation, programs were operated from the fall of 
1981 through December 1983 in five cities throughout the country. This 
demonstration has greatly expanded our knowledge about the implementation 
and operation of transitional -employment programs for this target popula- 
tion and has documented the effectiveness of such programs in enhancing the 
economic and social independence of mentally retarded young adults. Key 
findings from the evaluation effort include the following: 

1. The program did achieve its central objective of 
moving mentally retarded young adults into regular, 
unsubsidized employment. It was much more effec- 
tive during the "steady-state" period of program 
operations than during periods of start-up or 
phase-down, suggesting that ongoing programs would 
have even more favorable outcomes than did this 
short-run demonstration. 

2. The program was not effective in increasing overall 
employment activity among mentally retarded young 
adults, but it did substantially increase the pro- 
bability that they held regular jobs instead of 
workshop or activity-center jobs; thus, their earn- 
ings increased substantially, both in absolute 
terms and relative to the earnings gains estimated 
for employment programs targeted toward other dis- 
advantaged subgroups of the population. 

3. The program tended to have a greater net impact on 
the regular job-holding of those whose IQ scores 
were in the mild to moderate range than for those 
whose scores were in the borderline range, suggest- 
ing that programs should not "cream" from among the 
applicants they judge to be the most capable. 
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4. The program tended to be less effective with the 
women it served than with the men, especially 
during periods of operational transitions. 

5. The STETS model of transitional employment services 
can be implemented effectively at a seemingly 
reasonable operating cost relative to other program 
options and relative to the program*s benefits. 



This report on the impact evaluation and the benefit-cost analysis 



of the demonstration program consists of the following components: (1) a 
brief description of the rationale for the demonstration, and overviews of 
the STETS demonstration and the evaluation design, (2) a discussion of the 
success of the program in achieving its impact goals, (3) a comparison of 
the benefits and costs of the program, and (4) a review of the policy 
implications of the demonstration findings. A complementary report pre- 
pared by MDRC (Riccio and Price, 1984) discusses issues pertainii?^^ to 
program implementation and its potential replication. 

THE RATIONALE FOR THE I»fONSTRATION 

Transitional employment programs for mentally retarded persons have 
a relatively brief history, dating back only to the early 1970s, and none 
of the previous efforts has been subjected to as rigorous an evaluation as 
this one. Several related factors were especially influential in the 
initiative taken by the Manpower Demonstration Research Corporation to 
design the STETS demonstration. 



First, over the past 15 years, attitudes have changed considerably 



regarding the rights and abilities of mentally retarded and other handi- 
capped persons to participate more fully in society and to make substantial 
contributions to their own support. Among the prominent evidence of this 
shift are the Rehabilitation Act of 1973; provisions in the Vocational 
Education Act, the Comprehensive Employment and Training Act, and the Job 
Training Partnership Act that encouraged the participation of handicapped 
individuals in education and training programs; the Education for All 
Handicapped Children Act of 1975; and the 1980 and 1984 amendments to the 
Social Security Act whose purposes were to reduce the work disincentives 
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created by the Social Security Disability lR,arance and Supplemental 
Security Income programs. 

A second relevant factor is that, despite these federal efforts, a 

small proportion of persons who report a handicap are employed.^ Moreover, 

an even smaller proportion of the mentally retarded young adults are 

employed in regular, unsubsidized jobs. These persistently low employment 

rates, together with the substantial federal outlays for income support and 

2 

special education services to mentally retarded persons, have fostered a 
growing emphasis on ^intervention strategies, including transitional and 
supported employment. 

A third factor that fostered this demonstration was that two inde- 
pendent bodies of evidence suggested that transitional employment was a 
potentially effective way to facilitate the transition zt many mentally 
retarded young adults from school or workshop/activity centers into regular 
competitive employment. First, the results of the nt >nal Supported Work 
demonstration showed quite clearly that transitional employment programs 
could be effective in mitigating the employment problems of other seriously 
disadvantaged subgroups of the population, and that the effectiveness of 
the programs tended to be greater among the moie disadvantaged subgroups of 
the target populations served. Second, a number of relatively small tran- 
sitional employment programs for vTientally retarded adults, many of whom 
were young, have demonstrated the operational success of such efforts.^ 



1 

For example, according to the U.S. Bureau of the Census (1982), 
only about 25 percent of all persons reporting a handicap are employed. 
2 

In 1982, approximately 500,000 mentally retarded persons received 
SSI benefits totalling about $1.5-billion ( Social Security Bulletin > 1983); 
an estimated $225-million dollars were spent on education for 18- to 22- 
year-olds under P.L. 94-142. 
3 

See Bellamy and Melia (1984) for a discussion on the general 
rationale for the spiraling interest in these interventions. 
4 

See, for example, Hollister, Kemper, and Maynard (1984). 

5 

See, for example, Rusch (1980), Wehman (1981), Hill et al. (1985), 
Bailis et al. (1984), Vera Institute of Justice (1983), and Hill and Wehman 
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TUB STETS DKMCNSTRATION 

The STETS demonstration design reflects the influence of the 
national Supported Work demonstration, which emphasized transitional 
employment under close supervision (with peer-group support) and gradual 
increases in productivity demands. However, elements of the transitional 
and supported employment models that have evolved in the disability program 
arena have also been blended into the program model. 

The Target Population and Prograir. ^odel 

The STETS program model was designed specifically to serve the 
needs of 18- to 24-year-olds whose IQ scores ranged between 40 and 80, who 
had no work-disabling secondary handicaps, and who had limited prior work 
experience. It encompassed three phases of activity. Phase 1 consisted of 
initial training and support services that were generally provided in a 
low-stress work environment and which could include up to 500 hours of paid 
employment. Phase 2 was essentially a period of on-the-job training (sub- 
sidized or unsubsidized) in local firms and agencies. This period of 
graduated stress was to promote job performance that matched the perfor- 
mance of nondisabled workers in the same types of jobs. Together, Phases 1 
and 2 were to last no longer than 12 months of active time for any partic- 
ipant. By design. Phase 2 jobs were to roll over into postprogram jobs 
with the withdrawal of program support. Finally, Phase 3 consisted of 
follow-up services to workers who transitioned into unsubsidized, compe- 
titive employment. 

Participants and Program Services 

STETS was implemented in five sites — Cincinnati, Ohio; Los Angeles, 
California; New York, New York; St. Paul, Minnesota; and Tucson, Arizona — 
which were chosen both for their diversity in terms of program sponsors and 
geographical locations and for their project capabilities. In total, these 
five programs enrolled 284 participants between November 1981 and December 
1982; of this total, 58 percent of our evaluation sample were male, and 
half were from minority ethnic/racial groups. The majority (60 percent) 

XX 
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had IQ scores that indicated mild retardation, and another 12 percent had 
IQ scores that indicated moderate retardation. 

As was expected, the young adults who enrolled in the program 
exhibited a substantial degree of dependence on others. About 80 percent 
of them lived with their parents, and another 10 percent lived in super- 
vised settings. Less than 30 percent were able to manage their own 
finances. Nearly tt-jo-thirds were receiving some form of public assistance; 
one-third were receiving either Supplemental Security Income (SSI) or 
Social Security Disability Insurance (aSDI). 

Prior vocational experience was limited primarily to workshop and 
activity centers, and about one-third had had no type of work experience in 
the two years prior to enrollment. However, this group was obvious] y in- 
terested in making the transition to unsubsidized employment, as evidenced 
by the fact that 70 percent of the participants had worked or participated 
in an education or training program in the six months prior to applying to 
STETS. 

On average, young adults in our evaluation sample were enrolled in 
STETS for nearly 11 months, during which period they worked an average of 
710 hours in paid employment (370 hours in Phase 1 activities and 340 hours 
in Phase 2 activities). Participants in New York and Cincinnati had 
higher than average probabilities of entering Phase 2 employment and tended 
to have longer-than-average periods of paid employment. Also noteworthy is 
the fact that a disproportionate share of males entered Phase 2 (73 percent 
of males as compared with 57 percent of females). Forty-four percent of 
the program participants (51 percent of the males and 33 percent of the 
females) transitioned from the program into unsubsidized jobs (primarily in 
the for-profit sector) that paid, on average, about 10 percent above the 
minimum wage. 



1 

Those with Phase i paid employment (92 percent of all 
participants) worked an average of 400 hours in their Phase 1 jobs; those 
with Phase 2 paid employment (66 percent of all participants) worked an 
average of 513 hours in their Phase 2 jobs and 767 hours overall. 
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THE IMPACT EVALUATION KSI6N 

The STETS research plan was designed to address five basic 
questions: 

• Does STETS improve the labor-market performance of 
participants? 

• Does STETS participation help individuals lead more 
normal life-styles? 

• In what ways do the characteristics and experiences 
of participants or of the program influence the 
effectiveness of STETS? 

• Does STETS affect the use of alternative programs 
by participants? 

• Do the benefits of STETS exceed the costs? 

In order to address these questions, it was necessary to obtain 
data that would enable us to measure what happened to STETS participants 
during and subsequent to their program participation and what would have 
happened to them over this same time period had they not enrolled in 
STETS. We did so by adopting an experimental design whereby eligible 
applicants were assigned randomly either to an experimental group (and were 
enrolled in the program) or to a control group (and were sent back to their 
referral agencies). Implementing this procedure was feasible because STETS 
had been introduced into the sites as a new or expanded program of moderate 
size (a target of 40 to 55 slots per site) relative to the size of the 
areas' target populations. This random assignment of applicants is an 
especially important feature of the evaluation design: because it yielded 
groups that should be similar in all respects except for their participa- 
tion in STETS, it permitted relatively powerful tests of the effectiveness 
of STETS relative to the other options available to mentally retarded young 
adults in the demonstration sites. 

The Sanq>le and Data 

The final research sample consisted of 437 individuals — 226 experi- 
ment als and 211 control group members. The primary source of data on these 
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individuals came from in-person interviews that were administered to jample 
members or their proxies immediately after random assignment and again at 
6, 15^ and 22 months after random assignment.^ Because of the limited 
recall abilities of this target population, the surveys collected point-in- 
time data rather complete time-line information, such as has generally been 
collected for program evaluations that focus on other target groups. 

These interview data were supplemented with information from a 
variety of other sources. Information on program-service receipt was ob- 
tained from the demonstration's Management Inforraatiorx System, and program 
cost data were obtained from the demonstration's accounting system. Other 
sources of data that were systematically used in the study include com- 
munity service agencies, program Intake records, and speciexl work-site case 
studies. 

Analytic Approach 

The basic analytic approach used to estimate the impacts of the 
program entailed comparing the outcome measures for experimentals with 
those of controls by using regression analysis techniques. This approach 
enabled us to compute the overall impacts of the program and the impacts of 
the program on selected sample subgroups, while offering some gains in 
terms of the efficiency of the estimates. 

A benefit-cost methodology was undertaken to assess the economic 
efficiency of the program. The methodology involved assigning dollar 
values to estimated program impacts and to program costs, and then com- 
paritig these benefit and cost estimates to yield an estimate of the 
program's net present value—the difference between benefits and costs, 
denominated in base-period values. 

In applying these analytic approaches to the STETS data, our 
evaluation of the STETS demonstration has made two important methodological 



Response rates to these surveys were exceptionally high. Ninety- 
seven percent completed the baseline, and the respective completion rates 
for the 6-, 15- and 22-month surveys were 95, 91, and 89 percent. 
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contributions to evaluating transitional-employment programs for mentally 
retarded persons. First, it has documented that the availability of a 
control group can be very important for measuring the true effectiveness of 
employment md training interventions for this target population. Had we 
been limited to using the preprogram behavior of the participant group to 
estimate what the behavior of participants would have been in the follow-nip 
period had they not participated in STETS, we would have estimated substan- 
tially larger net program impacts than actually occurred. The reason is 
that, even in the absence of intervention, these mentally retarded young 
people tend both to increase their overall level of employment and to shift 
from noncompetitive employment settings (training jobs and workshop/ 
activity centers) into competitive employment. Nonexperimental methodolo- 
gies, which are adopted in most other evaluations, will tend to attribute 
some of these natural time trends to the effects of the program. The 
second methodological contribution of our evaluation is that mentally 
retarded young adults who enrolled in STETS were able to provide reasonably 
detailed and accurate information on their current circumstances and 
employmenc activities through in-person interviews. 

ESTIMATES OF IMPACTS 

The analysis focused on the impacts of STETS on four major areas: 
(1) labor-market behavior, (2) training and schooling, (3) public transfer 
use, and (4) economic status, independence, and life-style. 

Labor Market Behavior 

The evaluation clearly indicates that a STETS-type program can be 
expected to improve the post program employment prospects of mentally 
retarded young adults. As shown in Table 1, employment in regular jobs was 
significantly greater for experimental group members than for control group 
members in the postprogram observation period — that is, at months 15 and 
22. By month 22, experimentals were an average of 62 percent more likely 
than controls to be employed in a regular job (31 percent versus 19 
percent), and the regular-job earnings gains were proportionately larger 
($36 per week among experimentals versus $21 per week among all controls). 
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termined, and it is presumed that greater program efficiency is 
desirable. Equity questions are more difficult to answer. Benefit-cost 
analysis can (to some extent) determine the effect of a program on the 
distribution of resources, but contains no special criteria for judging 
whether a distributional change is desirable. Thus, addressing equity 
entails a more descriptive analysis, and conclusions must be based on a 
broader analysis of public policy and social concerns. 

The basic method used to determine economic efficiency is to assign 
dollar values to all estimated impacts and costs. These values are then 
summed together to yield an estimate of the program's net present value- 
that is, the difference between benefits and costs, where all dollar values 
are adjusted to reflect their value in a specific base period. A positive 
net present value indicates that the resources are being used ef- 
ficiently. 

While the net present value criterion can easily be stated, a high 
degree of uncertainty surrounds its estimation: program effects are 
measured imperfectly, and some cannot be estimated at all; uncertainties 
surround the values that should be placed on the specific program effects 
or costs; and the appropriate techniques necessary to aggregate individual 
benefits and costs inherently involve numerous approximations. Conse- 
quently, it is difficult to apply the net present value criterion to judge 
the economic efficiency of the program. 

Because of the error associated with any single estimate of net 
present value, much of the usefulness of benefit-cost analysis pertains to 
its comprehensiveness. The process of drawing together measures of the 
various inputs an^ outcomes and the general patterns that emerge from the 
attempts to assign relative values are often more useful than any specific 
estimate of net present value. For this reason, the analysis does not 
focus on a single net present value estimate but, instead, on a set of 
estimates. By examining the different assumptions, the underlying outcome 
estimates, and the techniques used to value outcomes, it is possible to 
identify those aspects of STETS and its evaluation that are most important 
in determining the overall findings— that is, which (if any) aspects, if 
changed, would change the basic nature of the findings. 
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The core of this approach is a benefit-cos t accounting framework 
that imposes a logical rigor on the analysis and serves as a guide for 
interpreting the results. The framework specifies a consistent method for 
valuing the diverse sets of effects. The approach used here is based on 
concepts similar to those that underlie the estimation of gross national 
product (GNP), It focuses on the net resource gain or loss induced by 
STETS as it was implemented in the demonstration* Essentially, the 
approach entails estimating the change in resources available because of 
STETS, and then valuing those resources at their market cost. Thus, for 
example, STETS-induced reductions in th^ use of sheltered workshops by 
experimentals enables society to reallocate some of the resources that 
would have been used to operate those programs. The market value of the 
saved resources is used as a measure of the value generated by the 
reduction in sheltered workshop use. In general, this valuation procedure 
is convenient to use and does not necessitate attempting to measure such 
difficult concepts as the willingness of society to pay for the various 
outcomes. 

While this procedure assigns a value to program effects, that value 

will be viewed differently by different groups. For example, experimentals 

who lose SSI benefits after obtaining a job will view the loss as a cost, 

while taxpayers will view it as a financial gain. The accounting framework 

captures these differences through three analytical perspectives: society 

2 

as a whole, program participants, and nonparticipants. The perspective of 
society as a whole abstracts from all of the redistributional aspects of 
STETS and focuses on its efficiency, since it considers only the use of 



1 

Of course, one possible reallocation would be to continue to 
operate the sheltered workshops at the same scale and to serve clients who 
would not have been served in the absence of STETS. 

2 

The nonparticipant group includes everyone in society who is not 
given the opportunity to participate in STETS. Thus, it encompasses much 
more than the control group, which comprises a very small part of the 
nonparticipant group. We prefer this terra to the more common terra 
"taxpayer group," because the participant group also pays taxes, and 
because not all of the effects on nonparticipants occur through the tax 
system. 



ERIC 



140 

29 



resources. Transfers of income between groups are assumed to cancel each 

other out in the social perspective—that is, a dollar of benefit or cost 

to one personals assumed to be equivalent to a dollar of benefit or cost to 

anyone else. The perspectives of participants and nonparticipants 

facilitate an analysis of the distributional consequences. For each group, 

the question for net present value is the same as it was for society as a 

whole: Does net present value (from that perspective) exceed zero? Do 

participants gain or lose, on average, from their participation? Are their 

earnings gains and increased independence sufficient to outweigh the losses 

of transfer benefits such as SSI? How are nonparticipants affected? Does 

2 

STETS require a net subsidy from nonparticipant taxpayers? 

All three perspectives mask differential effects on specific 
individuals or groups. The impacts of STETS are measured as averages and 
indicate the expected effect of STETS. Obviously, participants will differ 
in their response to STETS and may do better or worse than the statistical 
averages. In addition, individual nonparticipants will be affected 
differently. The employer who is able to hire a productive worker because 
of STETS will perceive the program differently than will an average 
taxpayer who helps to fund STETS. Therefore, the STETS benefit-cost 
estimates must be taken as indicative of the expected overall effects of 
STETS, viewed from a broad perspective that is appropriate for judging 
aggregate program performance. A more detailed analysis would be required 



1 

Of course, any resources consumed in transferring income would be 
counted as costs from the social perspective. 
2 

One analytically useful feature of using these three perspectives 
is that the sum of the net present values calculated from the participant 
and nonparticipant perspectives equals the net present value for the social 
perspective. This "adding-up" property is valid because participants and 
nonparticipants constitute mutually exclusive groups that, when combined, 
include all members of society. Therefore, transfers of income between 
these two groups cancel each other out in the social perspective, because 
the benefit to one group is assumed to be equal to the cost to the other. 
Benefits or costs that accrue to one group and that are not offset by 
corresponding costs or benefits to the other (e.g., increased work output) 
do not cancel out when added, and they thus represent net social benefits 
or costs. 
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to aiTSwer questions about whether specific types of individuals would 
benefit from enrolling in STETS. 

Because the benefits and costs of STETS occur over time, the 
analysis must compare streams of benefits with streams of costs. To 
simplify this task, we include in the accounting framework several pro- 
cedures for aggregating dollars in different time periods and for producing 
equivalent estimates of benefits and costs at a single point in time. To 
do so, it is necessary to account for differences in the value of benefits 
and costs across time periods due to inflation and to foregone interest 
earnings. The inflation differences are corrected by valuing all benefits 
and costs in 1982 dollars. Thus, differences between benefits and costs 
reflect real changes in resources, not changes in the nominal value of a 
dollar. The differences due to foregone interest reflect the fact that a 
benefit that occurs in the future is worth less than the same benefit that 

occurs today, because today *s savings could be invested and would earn 

2 

interest in the future. The procedure for adjusting for such differences 

is called "discounting," and its importance Is well established among the 

analytic literature (see, for example, Gramlich, 1981). The only 

uncertainty that remains pertains to the interest rate that should be used 

in discounting future benefits and costs. We assume a 5 percent real 

annual rate (that is, a rate calculated by netting out inflation) for our 

benchmark, and test the importance of this assumption by calculating 

3 

alternative estimates using real annual rates of 3 and 10 percent. 



1 

The subgroup analyses in Chapters IV through VII provide some 
information on the impacts of the program on individual types of 
participants. 

2 

For example, suppose that a $1,000 benefit occurs 10 years from 
now. Tlie issue would then be, what present value invested today would 
yield $1,000 ten years from now? If interest rates were, for example. 5 
percent, then this present value would equal $1,000 divided by (1 + .05)^^, 
or $614. Gramlich (1981) describes this process in more detail. 
3 

The Office of Management and Budget (1972) mandates a 10 percent 
discount rate for evaluating government programs. 

142 
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(Appendix C provides further information on the rationale and procedures 
for discounting.) 

Table VIII. 1 presents the benefit-cost accounting framework. The 
table lists the major impact components of STETS (regardless of whether we 
can value them) and suggests whether a component is, on average, a benefit, 
a cost, or neither from each of the three perspectives.^ The table also 
indicates what data sources are used to measure and value the effects or 
whether a particular effect is laft unmeasured. The next two sections 
discuss the separate cost and benefit components. 

Before proceeding, however, it is important that we review how the 
impact estimates presented in Chapters IV to VII are used in the benefit- 
cost analysis. These estimates indicate the effect of STETS on experi- 
mentals at 6, 15, and 22 months after randomization. These "point-in-time" 
estimates are adequate measures of the impacts of STETS, but are inadequate 
for the benefit-cost analysis, which requires information on the impacts of 
the program for the entire 22 months. In order to compare benefits and 
costs, we need to estimate the cumulative change in earnings, program use, 
transfer receipt, and other activities. In the absence of continuous dati 
on these activities, it is necessary to derive cumulative measures by 
interpolating between the point-in-time estimates. 

Any interpolation method involves some arbitrariness. We have 
chosen to interpolate linearly between the point estimates. This method is 
straightforward, and appears reasonable in that no alternative is clearly 
preferable. Thus, although all program impacts used in the benefit-cost 
analysis are inherently more imprecise than the specific impact estimates 
presented earlier, we feel that estimates of cumulative effects based on 
linear interpolations provide an accurate indication of the true magnitude 



1 

Whether an impact component will be a net benefit or cost is 
sometimes problematic. Table VIII.) reflects prior judgments about the 
value of components from the three perspectives. The treatment of all 
components in the final net present value calculations is of course 
determined by the estimated actual effects of STETS. 
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TABLE VIII. 1 

EXPECTED BENEFITS AND COSTS OF STETS BY ANALYTICAL PERSPECTIVE 



Analytical Perspective a 

Component Social Participant Nonpartlclpant Data Source 



Frograa Costs 

Project operations 
Payments to participants 



Work-activity centers 



Job-training programs 



Counseling services 



IV. Residential Situation 

Reduced use of: 



VII. Intaoglbles 

Preferences for work 



0 - A 

+ - A 



Central administration - 0 - ^ 

II. Output rtoduced by Participants 

Phase 1 and Phase 2 output + 0 + S 

Output forgone while In STETS - - 0 I,P 

Increased out-of-program output + + 0 

III. Other ProgrsMB 

Reduced use of: 

Sheltered workshops +0 + I»P 



+ 0 + I»P 



School +0 + I,P 



+ 0 + I»P 



Case-management services +0 + S 



+ 0 + E 



Sodal/recreatlonal services +0 + E 

Transportation services +0 + E 



+ 0 + 



Institutions 

Group homes + 0 * 

Poster homes + 0 

Seml-lndependent residential +0 + I»P 

programs 

V. Transfer Payaenta and Taxes 

Reduced SSI/SSDI 0 - + I»P 

Reduced other welfare 0 - + 

Reduced Medlcald/Medlcare 0 - + I»P 

Increased taxes 0 - + I»P 

VI. Travfer Adalnlstratlon 

Reduced use of SSI/SSDI +0 + I.P 

Reduced use of other welfare +0 + I»P 

Reduced use of Medlcald/Medlcare +0 + I»P 



+ + + U 



Increased self-sufficiency + + + ^ 

Increased variation In - " ^ 
participant Income 

Foregone nonmarket activity - " ^ ^ 

Increased Independent living + + + ^ 



NOTE: The Individual components are characterized from the three perspectives as being a net benefit 
(+), a net cost (-), or neither (0). 

*The codes used for the data sources are as follows: S-spedal study, I-lntervlew data, P-publlshed 
data source, A-STETS accounting system data, U-ltem not measured, and E-ltem measured but excluded 
because the effects of STETS were trivial. 
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of program impacts. (Details on the interpolation procedures are provided 
in Appendix C.) 

B. STETS PROGRAM COSTS 

The accounting framework disaggregates costs into three 
components: the operating costs of the projects, compensation paid to 
clients while they were in Phas 5 1 or Phase 2 activities, and central 
administrative costs. The operatxng and central administrative costs are 
paid by nonparticipants. Because these costs represent the value of the 
resources used to operate STETS, they also represent social costs. 
Participant compensation is treated as a transfer from nonparticipants to 
participants, because it represents a shift in resources from one group to 
another. 

!• Operating Costs 

During the 27 months of operations, the five projects served 284 
clients and reported operating costs of almost $2,500,000, implying average 
operating costs of $8,800 per client. However, for two reasons, this 
estimate is misleading for the benefit-cost analysis. First, it 
corresponds to all clients, and not to the group of 226 participants who 
were included in the research sample. Second, it includes costs that are 
attributable to the fact that STETS was a demonstration. 

The first problem can be corrected in part by adjusting for 
differences between experimentals and other clients in their length of 
program participation. We estimated the cost of serving an active client 



1 

Data on program costs and participation in STETS were obtained 
from demonstration accounting and client-tracking records. Because these 
records provided data for the full observation period, interpolation 
between point estimates was unnecessary for estimating the following: 
STETS program costs, STETS oayments to participants, and the value of 
output produced by participants while in Phase 1 and Phase 2 jobs. 
2 

The output produced in Phase 1 and Phase 2, which is related to 
this compensation, represents real resource gains. Section C discusses 
both this output and changes in output produced outside of STETS. 
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for a month and then multiplied this average cost by the observed average 
length of time during which experimentals were active in the 
demonstration. This method is accurate as long as the five demonstration 
projects provided the same level of service per month to experimentals as 
they did to clients who were not in the research sample* Because there is 
no evidence of such differences in service provision, we feel that the 
method is sound* 

It is more difficult t^ correct for thc^. special demonstration 
costs. Oar general rule was to include all costs of the demonstration as 
it was fielded, with two exceptions. First, we subtracted an estimate of 
research^related costs — the costs of finding and screening applicants who 
were ultimately assigned to the control group, of completing and processii'ig 
the research data collection forms, and of staff time spent with the 
researchers. Riccio and Price (1984) estimated that these costs 
constituted 5 percent of total project expenditures (including both 
operating and participant compensation expenditures). Because these 
research costs probably had little or no effect on the impacts of the 
program, we feel that they should be excluded from the benefit-cost 
analysis. 

We made the second exception to the principle of estimating 
operating costs as actually incurred in the demonstration because several 
demonstration-specific features made the observed costs abnormally high. 
In particular, the limited duration of the demonstration meant that the 
actual costs overrepresented the higher average costs of the initial 
project implementation and of the demonstration phase-down period. 
Additional costs were also incurred because projects found it necessary to 
take special precautions to deal with the funding uncertainties surrounding 
the demonstration itself. Riccio and Price (1984) discuss these problems 
and suggest that the costs incurred during a five-month "steady-state" 
period best represent the costs of operating STETS on an ongoing basis « 
This period (which covers slightly different months at each site) fell in 
mid- to late-1982, a period when enrollments were high and operations were 
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stable relative to earlier and later periods. We have i-sed average costs 
from these periods in the benefit-cost analysis. 

After making these two adjustments for research costs and the 
effects of start-up and budgetary uncertainties, we estimate that it costs 
$666 per month of active participation to provide STETS services.^ 
According to data collected by MDRC as part of its client-tracking system, 
experimentals were active for an average of 9.3 months. When discounted at 
a 5 percent real annual rate to the point of enrollment in STETS, the 
implied participation cost is $6,050 per participant. 

2. Participant Compensation 

Participant compensation includes the wages and fringe benefits 
that the five projects paid to participants in Phase 1 and Phase 2 
activities. It also includes wages paid by employers directly to 
participants in Phase 1 and Phase 2. The projects reported all these 
expenditures monthly to MDRC as part of the demonstration monitoring and 
accounting system. During the five months of the steady-state period, 
participant compensation averaged $341 per month of active participation 
(average compensation expenditures over the entire demonstration were $30 
lower than during the steady-state period). Given the estimated average 
length of active participation (9.3 months), this figure implies that 



1 

As mentioned in Chapter IV, employment impacts were greatest for 
persons who were served during the steady-state period, suggesting that the 
higher average costs during the demonstration start-up and phase-down 
periods occurred because the projects were establishing new procedures and 
dealing with inefficiencies due to small and changing scales, rather than 
because extra services were being provided. This finding further supports 
the use of steady-state costs in the benefit-cost analysis. 
2 

Riccio and Price (1984) reported costs on the basis of enrollment 
months rather than active months. Because some clients were inactive for 
part of their enrollment period, the cost per enrollment month will be 
lower than the cost per active month. However, the average length of 
participation will be correspondingly higher when measured in enrollment 
time rather than in active time. Thus, the cost per client (the product of 
the cost per month and the months of participation) is independent of how 
time is measured. (Appendix C discusses the cost-estimating procedures in 
greater detail.) 
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participants received Phase 1 and Phase 2 compensation worth $3,094 per 
participant when discounted to the time of enrollment. 

3, Central Administrative Costs 

Central administrative costs cover the activities necessary to 
administer the contracts with the five projects and to provide 
demonstration-wide coordination. MDRC performed this task in the 
demonstration, although state or federal agencies would probably assume 
this role in ongoing program. For example, a state vocational 

rehabilitation agency could fund the programs and would assume 
responsibility for audit and performance monitoring. Estimating central 
administrative costs was difficult in STETS because of the overlap between 
MDRC's monitoring and research activities. Their dual role in the 
demonstration meant that their costs exceeded the central administrative 
costs that would be incurred in an on-going program. In addition, they 
incurred substantial start-up costs as they selected the sites and helped 
them ope rationalize the program. Consequently, the demonstration 
experience does not provide an adequate guide to estimating future central 
administrative costs. Our benchmark estimate is that central 

administrative costs would be approximately $20 per month if STETS were 
operated in a fairly decentralized manner whereby most of the monitoring 
would focus on audit responsibilities and fairly straightforward 
performance measures. Costs would be higher if the central authority 
provided intensive monitoring or technical assistance. 

C. OUTPUT PRODUCED BY PARTICIPANTS 

The analysis of STETS-induced effects on output produced by 
participants distinguishes between goods and services produced by 
participants in Phase 1 and Phase 2 and those produced by them outside of 
STETS. These two types of output have different distributional 
consequences and necessitate using different estimation techniques. 
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!• Value of In-Program Output 

The value of ounput produced by participants in Phase 1 and Phase 2 
is an important program benefit. This output accrues to nonparticipants 
(and to ^ society) and has been very important in previous benefit-cost 
studies. One measure or the value of in-program output is the revenue' 
generated either by payments made by firms which used participant labor or 
from the sale of participant-produced goods. This valuation method 
provides a reasonable lower-bound estimate of the value of in-program 
output, because the output was actually purchased for this amount.^ 
However, the STETS sites did not pursue revenue-generating strategies as 
their primary goal; rather, they focused on securing placement and training 
for the STETS participants. Thus, revenue may seriously underestimate the 
actual value of output. Based on estimates by Riccio and Price (1984), 
revenue for the five-month steady-state period was over $108,000, or $131 
per active participant month. Thus, revenu''; offset almost 40 percent of 
participant compensation. 

As an alternative method f OiT es timating the value of the in— program 

output of participants, we conducted a series of work-activity case studies 

for 33 randomly selected experimentals. For each person, we estimated the 

net value of the output they produced during a two-week reference 
4 

period. This estimate was based on the alternative supplier's price of 



1 

For example, in the national Supported Work demonstration, the 
value of output offset approximately 65 percent of the social costs (see 
Kemper and Long, 1981, p. 269). 

2 

Revenue is a lower-bound estimate under the assumption that 
profit-seeking employers would not pay more than a product (participant 
labor service) was worth to them. Thus, the value of output should not be 
less than what was actually paid. Of course, this argument is weakened if 
altruism prompts employers to overpay because of their desire to support 
the STETS program and its participants. In either event, the direct 
estimates of the value of output (discussed in the text) offer a more 
accurate estimate of the resource value of participant output. 
3 

We also studied seven participants who were not included in the 
research sample. The results for these participants were similar to the 
results for the 33 experimentals (see Appendix C). 
4 

The studies were completed between September and December 1982, 
the period that is generally considered to be part of the "steady-state" 
period. 
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the participant output—that is, on the wages and fringe benefits that 
would have been paid by an employer to other workers to produce the output 
that was produced by the participant. This estimate assumes that employers 
can obtain additional labor at the wages paid to their regular employees. 
To obtain the net value added by participants, we subtracted from the 
alternative supplier's price an estimate of the costs of additional 
employer-provided supervision and reduced output from other workers from 
usijg the participant labor. 

These estimates of the net output are based on in-person interviews 
with the participants* supervisors, and rely on those supervisors* 
judgments about participant productivity, supervisory costs, the reactions 
of other staff, and the source and cost of alternative labor to perform the 
participants' tasks. These estimates are also subject to problems because 
of the difficulty in assessing whether the use of STETS-participant labor 
enabled the employer to Increase output. There were several instances 
where an employer took on a STETS participant and raised the quality of the 
output, rather than increasing the amount or price of that output. For 
example, one STETS participant was hired by a day-care program operator. 
The operator did not change prices or increase enrollments, but, instead, 
apparently used the STETS participant to increase the amount of attention 
given to the children in the day-care program. Our net value-of-output 
estimates do not adjust for such quality changes. Thus, our estimates may 
be too low in some cases in which quality changes occurred. 

Despite these limitationf!, the estimates of the net value of 
participant output represent useful indicies of the value of in-program 
output. They are based on careful, systematic case studies, and, where 
possible, incorporate actual wage, fringe-benefit, and production records 
in their derivation. The net-value-added estimates indicate that 
participants in the research sample produced output worth an average of 
$293 per saonth of active participation during Phase 1 and $503 per month 
during Phase 2. Given an estimated average length of active participation 
of 5.5 months in Phase 1 and 3.8 months in Phase 2, ihese figures imply a 
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total value of output, discounted to the time of enrollment, of .,3,434 per 
participant (an amount that would more than offset participants' in-program 
compensation). 

2. The Value of Out-of-Program Output 

STETS affected output produced outside of the program in two 
ways. As participants entered STETS, they gave up some alternative 
employment opportunities. Later, as they completed their STETS training, 
many participants were able to work more than would have been the case in 
the absence of STETS. These changes in output enter the benefit-cost 
analysis from the perspectives of society and participants. 

Participants will perceive foregone production as a cost and 
increases in production as benefits. These changes also enter the social 
perspective to the extent that they represent a net change in total 
output. It is generally assumed that this is the case. However, if 
participants displaced workers who would have otherwise held the jobs 
filled by participants, then the lost output of those other workers must be 
subtracted from the increased output of participants in order to calculate 
the net change in social resources. In the extreme, STETS may have simply 
shuffled workers among a fixed number of jobs, with no net increase in 
output. The participants would have had higher incomes, but at the expense 
of other workers who were displaced. 

At the other extreme, STETS may have enabled participant? to move 
from labor market in which an excess supply of labor existed to one in 
which an excess of demand existed. When participants leave the market in 
which an excess of supply exists, any jobs that they would have obtained 
are filled Ly workers who would have been unemployed otherwise. From the 
social perspective, this effect implies that no output is foregone by 
having participants enter STETS, and thus that the social cost of 
participation is zero. In this case, social benefits would equal the 
increase in participant earnings plus the increased earnings of 
nonparticipants who fill jobs vacated by participants. Of course, this 
result requires that participants not be placed back into the excess labor- 
supply market. 
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STETS jobs often seemed to be in markets in which an excess supply 
of labor existed* The demonstration was fielded during an economic 
recession, and the possibility of displacement seems relatively high, both 
on the jobs that they held and on those that they would have held. These 
indirect labor-market effects will affect the social value of the output 
produced in Phase 1 and Phase 2, as well as the social value of the output 
produced after leaving STETS. 

The net change in social output in the presence of displacement can 
be valued in several ways. We chose to estimate net present value under 
the assumption that no displacement occurred, and then to assess the 
importance of this assumptio** by calculating alternative estimates under 
the assumption of some displacement. This approach can be thought of as 
indicating the potential of STETS to increase social output, provided that 
raacroeconomic conditions are adjusted to take advantage of it. Moreover, 
given the absence of any empirical basis for estimating the extent of 
indirect labor-naarket effects (or even knowing the direction of their net 
effect), no clearly preferable alternative exists. 

Under these assumptions about indirect labor-market effects, the 
change in out-of-STETS output was estimated as the change in total 
compensation received by participants (i.e., gross wages plus fringe 
benefits). If the markets function competitively, then the actions of 
employers and workers will ensure that total compensation equals the value 
of workers' contribution to output. In addition, regulations pertaining to 
wage rates in sheltered workshops require that compensation in that sector 
reflect productivity. To estimate total compensation; we multiplied the 
after-tax earnings estimates derived from interview data by a factor that 
reflected tax-withholding and fringe-benefit rates. This factor indicates 
that total compensation is 45 percent larger than after-tax earnings. To 
ensure that the estimates correspond to out-of-program output, we used non- 
STETS earnings as our measure. 



1 

As described in Appendix C, gross earnings (i.e., before tax 
withholding) are 23 percent greater than after-tax earnings. In addition, 
fringe benefits for low-wage workers such as STETS participants were 
estimated to be 18 percent of gross earnings. 
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The resulting calculations indicate that at the 6-month poinc, when 
most experimentals were in STETS, the controls had $16.89 more per week in 
non-STETS total compensation than did experimentals. This differential was 
reversed at month 15, when experimentals earned $5.76 more total compen- 
sation per week than did controls. At month 22, when all experimentals had 
left STETS and all earnings were from non-STETS employment, the experi- 
mental-control differential in total non-STETS compensation had risen to 
$13.50 per week. 

These estimates, along with the interpolations, are shown in Figure 
VIII. 1 (a zero experimental-control difference was assumed for tho time of 
enrollment). The cumulative change in total compensation—which equals 
the value of out-of -program output— is shown as the shaded area. It 
indicates that, during the fifteen months after randomization, participants 
forewent non-STETS jobs in which they would have produced output worth $437 
per participant. During the seven months between month 15 and month 22, 
participants produced increased non-STETS output worth $290 per partici- 
pant. When discounted to the time of enrollment, these estimates imply 
$425 of output foregone per participant in the first fifteen months and a 
subsequent increase in output worth $268 per participant. 

We have used these two figures to approximate both the foregone 
output while participants were in STETS and the increased postprogram 
output. Of course, since most participants spent less than fifteen months 
in STETS, this approximation is fairly rough. However, despite the 
imprecision in this disaggregation, the sum of the cwo estimates is an 
accurate estimate of the net change in non-STETS output during the 22-month 
observation period. 

D. OTHER BENEFITS OF STETS 

While the primary objective of STKTS was to increase employment and 
earnings, the intervention also generated other important impacts. We 



1 

Details on the calculation of total compensation and the 
interpolation are provided in Appendix C. 
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FIGURE VIII.l 



CUMULATIVE IMPACT ON TOTAL COMPENSATION 
PER PARTICIPANT FOR THE 22-MONTH OBSERVATION PERIOD 
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ESnmTBD PROGRW VPKOS ON KEY OUTCWC >€ASURES 



Oubcqiie He»ures 



Horth 6 Month IS Month 22 

Control Cbntrol Control 

ExperlMental Qraup Estlmttad experimental Group Esttnkfted Experlwntal (ktxp Estimated 

QrooMean Mean L^pact Group Mean Mean Iwpact (k^^jp Mey^ Iwpact 



Peroent Ei^Ioyed In 11,8 10,7 1,1 26,2 16,8 9,*»* 31,0 19,1 11,9»» 
R«9jUr Obb^ 

Rei jent Employed In 67,8 ^5,2 22.6** H,8 ^3,6 1,2 H,7 «,7 1,0 
Rtld 3ob 

Ay«rage Weddy earnings $11,81 $9,81 $2,00 $26,90 $16,31 $ 10,59»* $ 36,36 $20,55 $ 15,81*» 
In Regular Oob 

Average ie^y Earnings $ 52,39 $ 25,93 $26,46" $37,91 $ 26,W $ 11,43»» $ W,79 $ 28,M $ 12,38»» 
In ^Pald 3ob 

TralnLig md SehooUng 

Percent In My Training 61,7 W,6 21,1»» 20,6 29.4 -7, 8* 16.6 29,1 -12, 5»* 

Peroef* in My Schooling 7,5 15,7 -6.2" 6,2 10,1 -3,9 8.0 11,4 -3,4 



PerOBft Rfloeivlng SSI 26,3 31.0 -4,7 33.1 40.7 -7,6" %,9 40.2 -5.3 
or SSDI 

Average Northly IncoMe $66.41 $^.59 $ -«,18 $91,35 $ 109.65 $ .18,M $ 99.27 $ 120,03 $-20.76 
ttm SSL or SSDI 

Pleroenfc Raoeivlng My 31,7 43.1 -11,4*» 44,5 51.5 -7,0» 49,6 52,0 -2,4 
Cash Tranefiers 

Average NDf«hly Incone $80,23 $99.98 $-19,75 $114.78 $ 139,72 $-23,94 $ 126,53 $ 136,08 $-9,55 
ttm Cash Transfers 

Average Mdy Personal $71,72 $ 50,94 $ 20,78" $ 67,22 $ 59.67 $7,55 $71.59 $ 62,39 $9,20 
Income^ 

NOTEi These reaUts were estUiated through ordinary least squM techniques. Estisiated l«pacts or selected blnvy and tniKated outoow 

■eaeures were generated using probtt and tottt analysis, respectively, with virtually identical rcwlts. (Tlwse results are presented in 
Appendix A to the report.) 

io^ « those that are neither tralning/nock-study nor woricslttp/actlvity-oenter Jobs. 

Wsonal InoM includes earnings, oaih transfer benefits WK, general assist^toe, Supplenental Seourlty Incow, «vj Social Seourlty 
Disability Inauranoe), and other regular souroes of InooM. 

^Statistically significant at the 10 percent level, two-tailed test, 
^^Statistically sicpificant at the 5 percent level, two^taUed test. 
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Furthermore, as shown in Figure 1, this postprogram increase in employment 
in regular jobs was roughly equal to the reduction in employment in work- 
shop/activity centers • The STETS experience tended not to affect the 
average postprogram incidence of holding non-workshop-training jobs. Thus, 
although the overall level of emplojrment was largely unchanged, very 
important compositional effects occurred* Overall, average earnings from 
all types of employment increased by 44 percent in month 22 ($41 per week 
for experiment als versus $28 per week for controls). 

As shown in Table 2, the postprogram (month 22) results vary 
substantially among subgroups of the sample. The St. Paul program clearly 
had the largest impact on the regular job-holding of participants (41 
percent of the experiment a Is, compared with 18 percent of the controls, 
held regular jobs). In terms of the differential impacts among subgroups 
defined by their personal characteristics, STETS seems to have been most 
effective for four groups: those with lower IQ scores and whose retard- 
ation has organic causes; older individuals; males; and those who were more 
independent, as evidenced by their living arrangements and money-management 
skills at the time they enrolled in STETS. 

Of these results, two sets are especially thought-provoking — the 
results by IQ level and the results by gender. Estimated program impacts 
on the probability of holding a regular job were essentially zero for those 
participants with borderline IQ levels, 12 percentage points for those 
whose IQ scores indicated mild retardation, and 28 percentage points for 
those whose IQ scores indicated moderate retardation. The net effect of 
these differential program results is that STETS tended to raise the 
employment prospects for the mild and moderate retardation subgroups from 
levels that In the absence of STETS would have been well below those for 
the borderline retarded group to levels that were roughly similar to those 
of the borderline group. 

The impact estimates for males and females show a clear pattern — 
STETS had substantial impacts on the employment and earnings of males but 
no impacts on the employment and earnings of females. For example, in 
Month 22, both male and female participants would have earned an average of 
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FIGURE I 



TRENDS IN TYPES OF JOBS HELD BY 
EMPLOYED EXPERIMENTALS AND CONTROLS 
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about $18 to $20 per week on regular jobs ($107 per week among those 
employed) in the absence of STETS, as evidenced by the earnings of the 
control group members. However, participating in STETS raised the average 
regular-job earnings of males by 144 percent, to $48 per veek ($134 per 
week among those employed). The average regular job earnings of females in 
the experimental group ware comparable to those of their control-group 
counterparts in month 22 (about $21 per week, on average). These differ- 
ential results for males and females seem to be related to the observed 
greater difficulty of the programs in placing females in Phase 2 program 
jobs, especially during the non-steady-state periods of program operations. 

Tralnlpg and Schoolli« 

The employment impacts that were estimated for STETS were hypothe- 
sized to lead to reductions in the use of other training programs and 
schooling. Such an outcome is important because many forms of training and 
schooling for mentally retarded young adults are expensive, long-term pro- 
grams that do not have a strong record of placing individuals into competi- 
tive employment. 

A negative program impact on schooling was evident at month 6 as 
the activities of experimentals were absorbed by STETS; a smaller negative 
impact persisted in the postprogram period. However, the pattern of 
impacts on training is more complicated. Although controls used training 
programs to a great extent during the early months after their application 
to STETS, experimentals used such programs to an ever greater extent during 
their in-program period, since STETS, itself, was a training program. 
However, as expected, STETS did substantially reduce the use of training 
(primarily workshop-related training) in the postprogram period by 43 
percent, an amount roughly equal to the postprogram increase in regular 
employment (see Table 1). Thus, the STETS-induced increase in employment 
did carry with it overall resource-cost savings from the reduced use of 
training programs and schooling. 
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Public Transfer Use 

The hypothesized employment impacts were expected to lead to 
reductions in the use of public transfers. Although some evidence suggests 
that STETS did reduce dependence on cash transfers, the results are sialler 
than were initially expected. The impacts shovm in Table 1 are reasonably 
large for the in-program period, with a 26 percent decrease in the receipt 
of any cash transfers (32 percent for experimentals versus 43 percent for 
controls). However, this impact faded over time, until no statistically 
significant impact was evident by month 22 (although the point estimate is 
a reduction of 3 percentage points in transfer use and an average of nearly 
$10 per month in benefits). 

It is noteworthy that relatively larger persistent reductions were 
estimated for the portion of cash transfers accounted for by Supplemental 
Security Income and Social Security Disability Insurance. (The percentage 
decrease in the receipt of these transfers hovered around 15 points.) In 
month 22, the $21 per-month average reduction in SSI/SSDI benefits offset 
about 40 percent of the average $54 per-^raonth earnings gains; at the same 
time, benefits from other cash transfers (primarily welfare benefits) 
increased by 50 percent, from $19 to $29 per month, due at ?east in part to 
the increased receipt of unemployment insurance benefits. 

The subgroups that were responsible for the small overall reduc- 
tions in cash transfers are generally those that experienced reductions in 
SSI/SSDI income. As shown in Table 2, these subgroups are often, but not 
always, the same subgroups th&t experienced the greatest gains in regular 
employment from participating in STETS. 

Econorlc Status^ Independence, and Life-Style 

Employment Impacts were also expected to influence socio-economic 
status, independence, and life-style. STETS did raise the total personal 
income of participants, but by less than would be indicated by the earnings 
gains, since the earnings gains coincided with reductions in cash transfer 
benefits, as was noted above. In the postprogram period, STETS increased 
total personal income by between $8 and $9 (see Table 1). Perhaps because 
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of the modest level of this increase in personal income (generally among 
the same groups who were moving into regular employment, as shown in Table 
2), we failed to detect any strong patterns of program impacts in the 
postprogram period on measures of independence and life-style. Admittedly, 
measures of independence and life-style are less well defined than are 
measures of the other outcomes under evaluation. However, it is likely 
that the l.icome gains for most experimentals were too small to effect 
measurable changes in other aspects of Independence or life-style; and, for 
those with more substantial income gains, the observation period may simply 
have been too short to expect that the gains would translate into long-term 
adjustments in these areas. 

RESULTS OF THE BSNEFIT-COST ANia.TSIS 

The benefit-cost analysis suggests that programs such as STETS can 
be a worthwhile social investment. On average, it cost $9,400 per partici- 
pant to provide the STETS intervention, of which $3,100 represents wages 
paid to the participants in Phase 1 and Phase 2 jobs.^ During the 22-raonth 
observation period, this investment yielded increases in output and 
reductions in the use of other programs by participants that offset about 
85 percent of this initial investment. Consequently, we observed a 
measured net social cost of about $1,000 per participant for the period 
covered by our data. However, trends observed for the impacts on earnings 
and the use of sheltered workshops suggest that benefits will persist and 
are likely to outweigh costs in the long-run. If the earnings gains and 
benefits from the use of reduced alternative programs continued undimin- 
ished for as little as seven months beyond the 22-month point, social 
benefits would exceed social costs. This finding is consistent with other 
successful training programs that incurred substantial up-front costs for 
training in order to generate long-run benefits. 
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This average cost per participant compares favorably with the 
average costs of other similar programs targeted toward mentally retarded 
populations and other disadvantaged groups (see Thornton and Maynard, 1985, 
for such a comparison). 
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In addition, the STETS investment created intangible benefits by 
increasing the overall employment and social opportunities available to the 
participants. While the evidence is limited, the fact that many partici- 
pants tended to remain voluntarily on their jobs suggests that STETS 
enhanced the quality of their lives. Furthermore, the increased income, 
regular job-holding, and social Interaction provided by STETS are also 
expected to represent benefits. 

Because of their increased earnings both during and after STETS, 
participants benefitted substantially from the intervention. On average, 
they received a net benefit during the 22~month observation period of 
$2,100 per participant, and this amount is expected to grow as the parti- 
cipants continue to work. 

From the perspective of nonparticipant taxpayers, STETS required a 
substantial investment of $3,100 per participant during the observation 
period. This group paid not only the program operating costs, but also the 
in-program participant-compensation expenditures. Approximately two-thirds 
of these costs were offset by output produced by participants while in 
STETS, by savings from the reduced use of sheltered workshops and other 
programs, and by savings in transfer payments (primarily SSI/SSDI). If 
these impacts persist at their 22-month levels, then the nonparticipant 
costs will be entirely recouped within four and a half years after random- 
ization. Even if measured benefits to nonparticipant taxpayers fall short 
of their costs, STETS may still be an attractive investment because it 
effectively achieves a widely stated goal of public policy — to increase the 
employment opportunities and performance of mentally retarded young adults. 

POLICY CONCLUSIONS 

The results of this evaluation demonstrate that many mentally 
retarded young adults can indeed perform adequately in competitive employ- 
ment situations, sometimes with minimal or no ongoing support services. 
More importantly, the results of this study indicate that transitional- 
employment services such as were provided by STETS can be very instrumental 
in helping mentally retarded young adults achieve their employment poten- 
tial. The main policy conclusions that evolve from this evaluation of the 
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STETS demonstration pertain to (1) the potential of STETS to litigate the 

employment and independence problems of mentally retarded young adults; (2) 

key features of the program design and targeting stratecies; and (3) the 
benefits and costs of the program. 

Program Potential 

STETS-type programs can be expected to have no impact on the over- 
all employment level of mentally retarded young adults who are recruited 
through methods similar to those used in this demonstration. However, they 
Will tend to move workers out of workshops and into competitive employment. 
While the net gain in unsubsidized employment attributable to STETS ii 
about 12 percentage points, this overall result represents aggregjce 
impacts and masks the substantially larger effects for selected subgrrups 
of the target population. Even at this apparently modest overall figure, 
the estimated impacts represent a movement into competitive employment of 
60 percent of those who would otherwise be in workshops or a-tivity 
centers. Furthermore, if one looks at the effects during the steady-state 
period of operations, when the programs successfully served both males and 
females, the overall employment impacts are about 25 to 30 percet t larger. 

These programs can be expected to reduce public outlays for SSI/ 
SSDI and education and training services substantially (especially 
sheltered workshops). However, partially offsetting increases in outlays 
for other cash transfer programs can also be expected. 

No clear evidence exists to indicate that aspect, of life-style or 
social well-being will improve. However, since participation in this 
program was voluntary, one can assume that most, if not all, participants 
placed some nonpecuniary value on obtaining regular employment. 

Prograa Design and Targeting 

Several noteworthy results from this evaluation ..ertatn to issues 
of program design and taiTgating. First, evidence clearly suggests that on- 
going programs might well achieve substantially greater success than was 
observed for the full SfETS demonstration, since we found much larger 
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estimated employment impacts when we compared the employment behavior of 
experimentals and controls who participated in the program during the 
"steady-state" period — an approximately five-month period when project 
operations were relatively stable. Second, Phase 2 activities (the 
supported employment and training experience within a competitive employ- 
ment setting) were a key part of the program treatment. Third, in select- 
ing participants, programs should not "cream" from among those who have no 
work-disabling secondary handicaps. As has been found in evaluations of 
employment and training programs targeted toward other segments of the 
population, IxETS was often very successful with those whose prospects for 
entering competitive employment ir the absence of the program were the 
lowest. One notable characteris :1c that identified such groups was an IQ 
score that indicated mild to moderate retardation. 

One caveat with respect to the targeting issue is the strikingly 
more favorable program impacts for males than for females, even though both 
groups are expected to have quite similar success rates in entering compe- 
titive employment in the absence of STETS. In view of the estimated 
importance of Phase 2 activities, the fact that a disproportionate share of 
female participants relative to males never entered Phase 2 employment may 
explain, in part, the lack of significant employment impacts for females. 
Obviously, further investigation of the nature of these greater difficul- 
ties in serving females is warranted. However, the results of this study 
suggest that a major factor that affects the ability of program operators 
to meet the greater challenge presented by female participants is simply 
whether their attention is diverted by major program operational changes, 
such as start-up or phase-down activities; in the steady-state, their 
success with females was only slightly lower than their success with males. 

Benefits aul Costs 

In the long-run, STETS-type programs are probably a socially 
justifiable investment, in as much as the total benefits (the net increase 
in the value of the goods and services produced) will outweigh the costs of 
the program. However, as is generally the case, taxpayers must be willing 
to make a substantial up-front investment that may not be repaid to society 
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or to the taxpayers for several years. Using the full STETS sample and 
assuming some modest decay (5 percent per year) in the postprogram results, 
this pay-off period to society is estimated to be about two and a half 
years, and the pay-off period to nonparticipants (tax payers) would be 
about four and a half years. However, if the effectiveness of ongoing 
programs were more similar to (he effectiveness of STETS during its 
"steady-state" operations, this pay-back period could be substantially 
shorter. 



Generalizabllity of the Findii^s 

In view of the evaluation design, the results of this study are 
quite robust and provide reliable estimates of what one would expect upon 
replicating the STETS demonstration. However, the study findings are 
limited in two respects. First, they are based on only five, judgmentally 
selected urban sites, where programs were specially designed and implemen- 
ted for this demonstration. We cannot be certain whether other program 
operators in other sites and who operate on-going programs under different 
social, political, and economic conditions would have similar experiences. 
(For example, economic conditions were generally poor during the period of 
this demonstration.) It is also problematic whether similar programs could 
be efficiently and effectively operated In rural areas or even in more 
dispersed labor markets. Second, the participants represent only a small 
portion of the areas' populations of mentally retarded young adults, with 
most of the screening and selection having occurred at the social service 
agencies that made referrals to STETS. The evaluation design did not 
enable us to estimate or, more importantly, to characterize the selection 
processes and participation decisions. Thus, we are unable to predict the 
total portion of the target population that could potentially be moved into 
competitive employment through the provision of STETS-type services. 

Despite these limitations with the study, the overall conclusion 
that transitional employment is an effective program for increasing com- 
petitive job-holding among mentally retarded young adults is quite clear. 
In fact, this program appears to be among the more effective employment and 
training initiatives that have been field-tested. For example, these 
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results compare favorably with the often-cited examples of successful 

employment and training programs (Supported Work for the long-term welfare- 

l 

dependent population and Job Corps), and they are much more favorable than 

2 

the results for youth-orientated initiatives other than Job Corps. 



1 

See Hollister et al. (1984), and Mallar et al. (1982). 

2 

See, for example, Maynard ( 1984), Bassi and Simms ( 1983), and 
Dickinson et al. (1984). 
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I. TRANSITIONAL EMPLOYMENT FOR MENTALLY 
RETARDED YOUNG ADULTS 



The growing interest in the concept of transitional employment 
reflects an increased recognition of the desirability and potential 
effectiveness of transitional -employment services in mitigating the 
employment problems of mentally retarded individuals. Twelve years ago, 
systematic efforts to help mentally retarded individuals obtain and 
maintain jobs in the competitive labor market were very limited. Since 
then, growing public support for hiring mentally retarded (and other 
handicapped) persons and advances in training and production technologies 
have made it easier for mentally retarded individuals to obtain and hold 
competitive jobs. Moreover, the outcomes of dozens of small prototype 
transitional-employment programs have helped change perceptions about the 
capacity of mentally retarded individuals to engage in competitive 
employment (see, for example, Bellamy, Horner, and Inman, 1979; Rusch and 
Mithaug, 1980; Wehman and McLaughlin, 1980; and Wehman, 1981). 

Of course, as the number of programs that offer the transitional- 
employment approach have grown, so has the demand for program account- 
ability. Legislators, administrators, mentally retarded persons, and their 
advocates all want to know the effectiveness of this type of inter- 
vention. Although competitive employment is now a widely accepted goal and 
although the feasibility of the approach has been demonstrated, concern 
remains about whether transitional-employment programs for mentally 
retarded persons should be expanded further. Questions remain about the 
magnitude of program effects and whether those effects are sufficiently 
large to justify program costs. Moreover, whether these programs can be 
replicated in new settings and whether they can operate at a size relevant 
to large-scale implementation are still major concerns. 

The purpose of the Structured Training and Employment Transitional 
Services (STETS) demonstration, funded at the national level by the U.S. 
Department of Labor, was to address these concerns by providing a rigorous 
test of transitional employment for mentally retarded young adults. This 
multi-year project, which was overseen by the Manpower Demonstration 
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Research Corporation (MDRC), operated transitional-employment programs in 
five cities nationwide and included a rigorous evaluation that followed a 
sample of almost 450 experimental and control-group members for twenty-two 
months after program application* 

The large-scale, rigorous evaluation design and the relatively long 
follow-up period distinguish the STETS demonstration as the most 
comprehensive test of transitional employment yet undertaken. It provides 
powerful information on the three principal policy issues surrounding 
transitional employment: 

1« Xmplenentation. Can the STETS model be implemented on 
a policy-relevant scale? VHiat are the advantages .and 
disadvantages associated with alternative variations of 
this basic model? 

2. Impacts* Does STETS improve the labor-market 
performance of its participants (compared with what it 
would have been in the absence of STETS)? Does STETS 
affect its participants* use of alternative training, 
education, social service, and income-support 
programs? Does STETS participation help persons lead 
life-styles that are more typical of the population 
norm? Is the program more effective for participants 
who exhibit particular characteristics or who have had 
certain experiences? 

3. Heneflts and Costs. What does it cost to operate a 
trans.'.tional-employment program on a policy-relevant 
scale? Are the outcomes sufficiently large to justify 
these costs? 

The implementation issue was analyzed by Bangser and Price (1982), who 
examined how the STETS projects were established, and by Riccio and Price 
(1984), who examined the demonstration program operations and developed 
suggestions for replicating the model. The impact and benefit-cost 
analyses are presented in this report. 

As shown in this report, STETS did succeed in improving s'.he 

postprogram employment prospects of mentally retarded young adults. At a 

point nearly two years after they entered the program and, on average, a 

year after they left it, former participants were 62 percent more likely to 
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hold a regular job than had they not entered the program, and the earnings 
gains associated with those jobs were proportionately greater* This 
increase in regular job-holding was largely concomitant with a reduction in 
workshop-related activities* Primarily because of these results, we 
estimated substantial social benefits from STETS* These benefits were 
large enough that their value would exceed program costs from the 
perspective of society if the observed trends were to continue for as 
little as seven months beyond the observation period* For participants, 
benefits clearly exceeded costs in the observation period. For 
nonparticipants, who paid the taxes to fund STETS, the point at which 
benefits might exceed costs is difficult to project, but the continuing 
savings to nonparticipants from the reduced use of sheltered workshops 
suggests that a few more years of savings at the observed rate would be 
required* Our findings in other areas (primarily transfer program use, 
independence, and life-style) are less conclusive. The reasons are 
unclear, but it may well be that behavioral changes in these areas require 
a loiter observation period in which employment and earnings gains are 
consolidated and are perceived as long-term gains. 

Chapter II presents the key elements of the demonstration design. 
In particular, it describes the basic STETS program model, the research 
focus, and data collection design. Chapter III explains how the design was 
implemented and describes the demonstration sample members, the success of 
the research data collection effort, and the analytical methods used in the 
evaluation. Chapters IV through VII examine the specific impacts of the 
STETS intervention. Each of these impact-analysis chapters begins by 
describing the expected activities of experimentals had they not entered 
STETS, and then analyzes the impact of STETS both overall and among 
specific subgroups. Chapter IV discusses the impacts of the program on 
employment, earnings, and other labor-market behavior; Chapter V discusses 
the results on training and schooling; Chapter VI presents the findings on 
public assistance dependence; and Chapter VII addresses the estimated 
impacts on economic status, independence, and life-style. Chapter VIII 
assesses the benefits and costs of STETS. Finally, Chapter IX summarises 
the various results of the impact and benefit-cost analyses and discusses 
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their implications in terms of public policies targeted toward this 
population of mentally retarded young adults. Additional details on 
specific impact estimates, survey and data collection issues, and benefit- 
cost evaluation procedures are provided in Appendices A, B, and C, 
res pectively. 

The remainder of this introductory chapter reviews the policy 
context of the demonstration. It begins by reviewing the evolution of 
public policy toward handicapped persons , particularly mentally retarded 
individuals. It then discusses the role of transitional services in 
helping disabled persons become economically and socially self-sufficient. 

A. PUBLIC POLICY, EMPLOYMENT, AND MENTALLY RETARDED PERSONS 

Until the late 1960s and early 1970s, mentally retarded persons had 
little opportunity to enter the mainstream of society. The more severely 
disabled persons were particularly likely to be segregated from their 
nondisabled peers. Former HEW Secretary Joseph Calif ano characterized the 
situation by noting — 

For decades, handicapped Americans have been an oppressed 
and, all too often, a hidden minority, subjected to 
unconscionable discrimination, beset by demoralizing 
indignities, detoured out of the mainstream of American 
life and unable to secure their rightful role as full and 
independent citizens. 

In this environment, little attention was paid to employment 
opportunities for mentally retarded persons. Education programs for this 
population were oriented little or not at all toward vocational training or 
employment (Moss, 1980). Furthermore, although sheltered workshops dated 
from at least the raid-1840s, community workshops specifically for mentally 
retarded adults were relatively rare until the 1950s. And those that were 
implemented were operated as if competitive employment were Impossible for 
virtually all mentally retarded persons. 
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Remarks made on April 29, 1977 (see Mayer, 1982). 
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This situation changed dramatically in the 1970s, In response to 
growing advocacy by and for disabled persons, several landmark pieces of 
legislation were passed that reflected a major change in policy and the 
recognition of the rights and abilities of these members of society. The 
Rehabilitation Act of 1973 (P.L. 93-112) and its amendments prohibited 
discrimination against the disabled in programs and activities sponsored 
under federal financial assistance. The Vocational Education Act (P.L. 
94-482) and provisions of the Comprehensive Employment and Training Act 
(P.L. 94-524) encouraged the participation of handicapped persons in 
vocational training and employment programs, while the Revenue Act of 1978 
provided tax credits for hiring handicapped workers. In addition, a number 
of court cases and growing pressure from advocates (and state budgets) led 
to the deinstitutionalization of many mentally retarded adults (see 
Scheerenberger, 1981), 

The centerpiece of this body of legislation is the Education for 
All Handicapped Children Act of 1975 (P.L. 94-142). This law, sometimes 
referred to as the "Bill of Rights for the Handicapped," led to the rapid 
expansion of special-education facilities, staff, and curricula. In 
particular, it requires that all handicapped children be provided with a 
free and appropriate education, as well as with the necessary related 
services (such as transportation, speech pathology, and psychological 
assistance). Thus, all handicapped persons from ages 3 through 21 are to 
be provided with an appropriate educational program, regardless of the 
nature or severity of their handicap. The legislation also requires that, 
to the extent appropriate, the education be provided in the least 
restrictive environment—that is, handicapped students should receive 
education in the same environment as nonhandicapped children, unless they 
are unable to benefit from the education in such a setting. 



In addition to prohibiting discrimination (Section 504), this act 
also mandated that architectural barriers be eliminated (Section 502), and 
that most federal contractors take affirmative action to employ the 
handicapped. However, although the Act was passed in 1973, the 
implementing regulations (45 CFR 84.1 et seq.) were not published until 
197 7 . 
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Through these laws, society has come to recognize the abilities and 
contributions of mentally retarded persons* However, the actual 
implementation of these laws has also left us with a renewed realization of 
the severity of the problems facing mentally retarded persons and the 
difficulties inherent in developing effective public policy for dealing 
with them. Even when President Ford signed the Education for All 
Handicapped Children Act, he remarked, "Unfortunately, this bill promises 
more than the federal government can deliver." Since then, federal 
appropriations have fallen well below authorized levels , and state and 
local governments have been struggling to provide the required special 
education services. Furthermore, particularly important to this study, 
funding adult services has become even more problematic. State departments 
of developmental disabilities and vocational rehabilitation face severe 
funding constraints and generally do not have the resources available to 
provide the long-range services and individualized programming required by 
mentally retarded individuals to become and remain self-sufficient. In 
terms of employment, sheltered workshops remain the major alternative 
available, and these facilities are typically crowded and often have long 
waiting lists. In addition, they have traditionally been characterized by 
low wages and slow movement toward nonsheltered employment (U.S. Department 
of Labor, 1979). 

The goal of public policy during the next ten years will be to 
maximize the efficiency of this service system so as to reduce the 
discrepancy between the promises of earlier legislation and the reality of 
current adult services. The constituency for these adult services is 
already large and continues to grow. In 1976, over 88,000 mentally 
retarded persons participated in sheltered workshops (U.S. Department of 
Labor, 1979). Many of these workshop participants want and do have the 
capacity to work in regular jobs. In addition, the schools will graduate 
many more mentally retarded young adults who can be expected to seek jobs 
as they leave special education programs that have stressed employment. 
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In school year 1982-1983, 24,000 mentally retarded students 
between the ages of 18 and 21 were being served under P.L. 94-142 (U.S. 
Department of Education, 1984a). 




The goal is to establish efficient programs and policies that enable 
mentally retarded persons (as well as persons with other disabilities) to 
participate In the mainstream of society to the fullest extent possible* 

B* TRANSITIONAL EMPLOYMENT Atm RELATED SERVICES 

Transitional-employment programs represent one promising vehicle 
for enabling mentally retarded persons to become more self-sufficient* As 
noted earlier, these programs have developed rapidly during the last ten 
years, from a few small prototype efforts to several large statewide 
programs* Before we describe the STETS model of transitional employment 
and our evaluation of its effects, it is useful first to define the concept 
of transitional employment, and then to review previous experience in 
implementing the concept and the research on its effectiveness. 

!• What is Transitional Employment? 

The term "transitional employment" has been applied to programs 
that differ widely in their structure and in the specific manner in which 
they provide training. Nevertheless, these programs share several common 
features. A recent planning meeting at the U.S. Department of Education 
(1984b) defined transitional -employment programs (TEPs) as follows — 

TEPs are short-term interventions that lead to the 
employment of service consumers. These programs aim to 
achieve employment in competitive jobs at or above the 
minimum wage, subsidized only by temporary job credits and 
similar incentives to employers to hire people with 
disabilities. TEPs emphasize employment in regular work 
places where non-disabled people work. They normally 
involve a goal of full time work (35+ hours per week) at 
the close of training. 



1 

Some interest has als^ been expressed recently in "supported 
employment." This model, v/hich emphasizes permanent or long-term 
subsidized employment, is intended to serve severely handicapped 
individuals who may never be able to hold an unsubsidized competitive 
job. However, it does emphasize work in settings (or enclaves) in which 
the individuals perform productive work and inte. ict with nonhandicapped 
persons. Adding to the possible confusion is the term "supported work," 
which has been used to describe transitional-employnent programs for both 
mentally retarded persons (see, for example, Wehman and Kregel, 1984) and 
other disadvantaged populations (see, Hollister, Kemper, and Maynard. 
1904). ^ • 
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Three aspects of this definition should be emphasized. First, the 
essence of transitional employment is that it is a short-terra inter- 
vention* Its purpose is to provide a bridge to employment, not permanent 
support for employment. The objective is to provide intensive training and 
job place'^ent in an effort to move persons into competitive employment, and 
to enable them to remain employed and become self-sufficient (although it 
is recognized that some persons will lose their jobs and may again need 
program services). 

In order to adhere to the short-term orientation and to promote 
client independence, trausitional-employment programs follow a plan whereby 
each client's training is gradually phased out. In phasing out the formal 
training activities, the programs continue to provide a variety of support 
services, including counseling, crisis intervention, and, if necessary, 
additional training. These follow-up services may, for example, help 
clients cope with a new manager, deal with disillusionment or confusion 
about the job in particular or working in general, handle relationships 
with co-workers, and adapt to new job tasks. Moreover, the services may 
also help an employer train supervisors to work with handicapped 
individuals or to restructure job tasks to fit the abilities of disabled 
workers. 

The length of time during which clients receive these follow-up 
services varies. Some programs (such as those operated by Moss, 1980, and 
Hill and Wehman, 1983) do not formally terminate services to clients. The 
programs continue to provide follow-up services long after a person is 
trained and placed in a competitive job. These services are often minimal 
(Moss estimated that long-term follow-up services cost $10 per client 
month), but they provide support that is deemed necessary to ensure long- 
term employment. Other programs adhere more strictly to the short-term 
objective of transitional employment, but they usually provide some 
placement or counseling services to former clients. 

Second, the definition of transitional employment embodies an 
emphasis on procuring work in the unsubsiaized, competitive job market. 
This focus reflects the goal of the programs to move persons into the 
mainstream of society. It is expected that these types of jobs will 
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provide the interactions with nonhandicapped peers and the earnings 
necessary to integrate persons into normal society. In comparison, many 
sheltered workshops generate employment opportunities, but offer little 
interaction with nonhandicapped persons and, in general, very low wages 

Third, most transitional employment for mentally retarded persons 
is oriented toward providing some training on the job that the person is 
expected to keep. By providing training in the work setting, programs try 
to minimize problems that may be experienced by trainees in generalizing 
newly acquired skills to different settings. This focus also enables 
traasitional-employment programs to work with job supervisors and co- 
workers in helping to establish the trainee in the workplace. Furthermore, 
training on the job means that the skills and behaviors that are taught can 
be tailored precisely to the trainee's job tasks and work environment. 

As part of the training, transitional-employment programs focus not 
only on job-specific work skills but also on social skills. Research has 
shown that social skills are as important as work abilities in determining 
job retention (see Gold, 1973; Wehman, 1975; and Rusch, 1979). Thus, 
programs provide assistance and guidance in self-care and appearance, 
relations with co-workers, transportation, functional reading, and 
communication. Programs also provide social support services that range 
from assistance in managing finances to assistance in obtaining housing and 
interacting with public assistance and other social-service programs. 

2« Experience with Transitional -Employment Programs 

The first efforts to implement a transitional-employment program, 
as defined here, were university-based programs, some of which include the 
university food-service program operated by James Moss at the University of 
Washington; the University of Illinois Food Service Training Programs 
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In 1976 (the most recent year for which comprehensive data are 
available), the average hourly wage rate in cert-^ied workshops was $0.81; 
the average for persons in work activity centers was only $0.43 (U.S. 
Department of Labor, 1979). At that time, the federal minimum wage was 
$2.30 per hour. 



directed by Frank Rusch; Project Employability , directed by Paul Wehman in 
Richmond, Virginia; project EARN, directed by Paul Bates at Southern 
Illinois University; and the Mid-Nebraska Mentally Retarded Services 
Program directed by Robert Schalock and Harry Drake in Hastings, 
Nebraska. These and other, similar programs concentrated on developing 
appropriate training methods for mentally retarded individuals and on 
establishing the feasibility of transitional-employment programs. These 
efforts tended to be small, often training fewer than ten individuals at a 
time, and generally required intensive involvement by highly trained staff. 

In addition to these university-based programs, a number of 
transitional-employment training programs have been operated by foundations 
and private organizations, including those operated by the Association for 
Habilitation and Employment of the Developmentally Disabled (AHEDD) in 
Harrisburg, Pennsylvania, by Job Path in New York City, by the Menninger 
Foundation in Kansas, and by Transitional Emplo3nnent Enterprises (TEE) in 
Boston, Massachusetts. These efforts were generally larger than the 
university-based programs, tended to serve less severely disabled 
individuals, and were more transitional in nature. These programs 
generally developed their own training protocols, often expanding upon and 
modifying techniques developed in employment and training programs for more 
general populations. 

In addition to these local programs, the Association for Retarded 
Citizens of the United States (ARC-US) has operated a national on-the-job 
training (OJT) program funded by the Department of Labor (see Stumbaugh, 
1982). This program provides a very modest intervention whose purpose is 
to increase incentives to employers to hire mentally retarded workers. The 
program offers eight weeks of wage subsidies in order to reimburse 
employers partially for the costs of on-the-job training provided to newly 
hired mentally retarded workers. In 1984, this program served 2,300 
clients and had a budget of over $1. 16-million. It has served over 30,000 
workers since its inception in 1967. This program may be limited by the 
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Moss (1980) and V7ehman (1981) describe i:be experience of these 
early university-based programs. 
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short duration of the intervention and by a lack of trained staff to help 
the employers provide on-the-job training. However, the program's wage 
subsidies have been used widely by more intensive transitional-employment 
programs to supplement their efforts. 

Another national program, which can serve as a supplement to trans- 
itional-employment programs, is the Targeted Jobs Tax Credit. This program 
provides a tax credit to employers who hire handicapped workers (or workers 
who have other specified employment difficulties). The credit totals as 
much as half of a worker's wages in his or her first year of employment and 
a quarter of the wages in the second year.^ Many transitional-employment 
programs have used this credit and the ARC-US OJT program as inducements to 
employers to hire program participants. 

Finally, the Comprehensive Employment and Training Act (CETA) 
identified handicapped persons as a special population and provided some 
mentally retarded persons with a variety of training and work experi- 
ences. In feet, several of the transitional-employment programs operated 
by foundations and nonprofit organizations used CETA funding to support 
their efforts. To aome extent, this type of support has continued under 
the Job Training Partnership Act ^(JTPA), which designates handicapped 
Individuals as a special population. 

Evidence on the potential value of transitional-employment programs 
also comes from some programs that have enrolled populations other than 
mentally retarded persons. The national Supported Work demonstration was 
particularly influential In shaping the STETS demonstration. That demon- 
stration, which was targeted toward long-term AFDC recipients, ex-addlcts, 
ex-offenders, and disadvantaged youths, found that the effectiveness of 
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The celling on the credit and the fact that it reduces the 
employee s regular deduction for employee wages imply that the credit is 
worth between $900 and $2,580 per worker in the first year (depending on 
the employer s tax bracket). It is reduced by about one-half in the second 
year of employment. 
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In some cases, transitional -employment program operators have 
reported that the emphasis of JTPA on local decision-making facilitates 
their efforts to secure funding. 
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transitional employment varied by target group: the AFDC recipients, who 
had the lowest prospects of securing jobs in the absence of the 
intervention but who were highly motivated, experienced long-run 
improvements in employment, but the labor-market impacts for the other, 
less disadvantaged, groups were small or zero (see MDRC, 1980, or 
Hollister, Kemper, and Maynard, 1984). This finding influenced MDRC in its 
decision to develop the STETS demonstration: because mentally retarded 
workers were severely disadvantaged but highly motivated, the expectation 
was that transitional services could be a particularly effective vehicle 
for improving their labor-market opportunities and actual employment. 

This program history suggests that transitional-employment programs 
are feasible, and that they may provide an effective vehicle for enabling 
mentally retarded young adults to become economically and socially self- 
sufficient. However, because of the general lack of rigorous evaluations 
of prior programs, it is difficult to judge their merits or to assess their 
appropriate role in public policy directed toward mentally retarded 
persons. While all previous programs have been successful at placing sorae 
mentally retarded individuals in competitive jobs, the costs of doing so 
have not been generally established. Moreover, it is unclear what would 
have happened to clients in the absence of these programs. Thus, while 
apparently successful, the magnitude of the program effects and the cost- 
effectiveness of the programs themselves are unknown. 

Despite the paucity of rigorous evaluations, some in-depth studies 
have suggested that transitional-employment programs for mentally retarded 
persons may be cost-effective. In particular, the work of Hill and Wehman 
(1983) and Hill et dl. (1985) suggests that Project Employability (a trans- 
itional-employment program for moderat'ely and severely retarded persons) 
generated benefits that exceeded costs by a substantial margin. In 
addition, program assessments made in Washington and Massachusetts have 
convinced those otates to expand their transitional-employment efforts 
(see, in particular, O^Neill and Associates, 1983, for a discussion of 
activities in the state of Washington). In addition, the Social Security 
Administration was sufficiently confident in this approach thai it has 
funded a demonstration to test the efficacy of transitional employment for 
mentally retarded SSI recipients. 



12 

68 



II. THE STETS DEMONSTRATION 



In view of the evidence from a number of small demonstration and 
program efforts which suggests that transitional-employment and training 
services can benefit mentally retarded young adults, the Manpower 
Demonstration Research Corporation (MDRC) assumed the task of designing and 
operating a demonstration to determine (1) how a set of employment-oriented 
transitional services could operate in practice on a scale previously 
attempted only by one organization (Job Path in New York City), and (2) how 
effective those services could be in placing the target population in 
unsubsidized employment. This chapter presents an overview of the 
demonstration design as it pertains to the impact and benefit-cost 
evaluations. It then describes the research issues that underlie this 
report and the data collection design necessary to evaluate those issues. 

A. THE DEMONSTRATION MODEL 

The natural tensions that arise from underlying programmatic and 
policy concerns affected the ultimate design of the STETS demonstration. 
On the one hand, efficient tests of particular program models necessitate 
that demonstration programs be designed and implemented with close 
adherence to specific implementation guidelines. On the other hand, a 
demonstration design that calls for implementing specific program models in 
a rigid manner precludes gaining substantive knowledge from affording 
program operators a reasonable degree of autonomy in designing and 
operati.ig programs that meet the established guidelines and objectives. 
The issues underlying the STETS demonstration were such that the optimal 
design fell somewhere between a design whose purpose was to test very 
specific procrair models (which would enable one to learn a great deal about 
a very limited set of program and policy questions) and a design whose 
purpose was to examine a variety of program models and implementation 
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Detailed information on the design and implementation of the STETS 
demonstration program is contained in MDRC's own implementation report 
(Riccio and Price, 1984). ^ 
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strategies designed to address established policy objectives (which would 
perndt a less in~depth analysis of a much broader set of programmatic and 
operational options) • 

This intermediate strategy was suggested by three factors* First, 
the overall objective of the program was clear from the start — to place 
mentally retarded young adults in competitive employment • Second, the 
wealth of previous experience with mentally retarded individuals and other 
groups who experience persistent employment problems suggested an 
employment strategy based on transitional services — services that might 
include social and world-of-work training, as well as job-skills 
trainings And, third, as a result of funding options, future programs are 
likely to be operated by local service organizations that tailor centrally 
developed plans to local needs and service availability^ Thus, the 
approach adopted by MDRC was to specify the STETS program in terms of (1) 
the definition of program phases, their general content, and their time 
parameters, (2) project selection criteria, and (3) client eligibility 
criteria* Within these constraints and guidelines, the individual projects 
had some latitude to implement their programs as they saw appropriate* 

1 

!• Program Phases 

The program experience for clients consisted of three sequential 
phases: (1) assessment and work-readiness training; (2) transitional jobs; 
and (3) follow-up support services* 

Phase 1; Assessment and Work-Readiness Training * Phase 1 of the 
program combined training and support services in a low-stress environment, 
the goal of which was to help participants begin to develop the basic work 
habits, skills, and attitudes necessary for placement into more demanding 
work settings* This preliminary stage, which was limited to 500 hours of 
paid employment 3 occurred in either a sheltered or a nonsheltered work 
setting, but, in all cases, the participant's wages were paid by «he 
project* 



1 

This section and the next borrows freely from Riccio and Price 
(1984), pp* 11-12 and pp* 21-23* 
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During Phase 1, participants were to engage in at least 20 hours of 
productive work weekly, with additional time spent, as necessary, in other 
activities that would enable them to develop the behavior and knowledge 
required in Phase 2 employment positions. Based on these activities, 
program operators were able to assess the abilities and interests of 
mentally retarded workers, a process which is considered essential for 
identifying necessary support and training services. The activities also 
provided information to help staff place participants into appropriate jobs 
in subsequent phases of the program. 

Phase 2; Transitional Jobs. At all sites. Phase 2 was essentially 
a period of "on-the-job" training in local firms and agencies. During this 
stage, participants were placed in nonsheltered positions that required at 
least 30 hours of work per week, and in which, over time, the levels of 
stress and responsibility were to approach those found in competitive 
jobs. In developing Phase 2 job slots, prograias emphasized positions that 
would lead to regular employment. Wages were paid by either the project or 
the employers and, in many cases, by a combination of the two. The STETS 
program provided workers in Phase 2 with counseling and other support 
services, and it helped the line supervisors at the host company conduct 
the training and necessary monitoring activities. 

Because STETS was to provide a relatively quick transition to 
employment, MDRC guidelines limited paid participation to 12 months during 
Phases 1 and 2 combined. To compensate for periods of inactivity caused by 
participant- or program-related problems, MDRC guidelines allowed 
participation to span a 15-raonth calendar period. 

^^^^^ ^' Postplaceme nt Support Services . The third phase of 

program participation began after participants had secured regular 
employment. According to MDRC guidelines, Phase 3 began when each of the 
following conditions was met: 



1. The employer was not receiving a financial subsidy from 
the program. 

2. The program had substantially reduced counseling and 
other services to both the participant and the employer. 
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3. The staff and the employer considered the participant to 
be a regular member of the workforce, rather than a 
trainee. 

The purpose of this phase of program services was to ensure an orderly 
transition to work by tracking the progress of participants, by providing 
up to six months of postplacement support services, and, if necessary, by 
developing linkages with other local service agencies. 

2. Pro:^ect Selection Criteria 

MDRC selected local program operators on the basis of two types of 
considerations — diversity and capability. In terms of diversity, the major 
elements Included geography and the nature of the local project organiza- 
tion. Geographic diversity entailed selecting sites that represented a 
variety of urban settings. To attain organizational diversity, MDRC 
sought projects from among traditional and modified sheltered workshops, 
state agencies, and nonprofit training programs. 

In terms of project capability, MDRC considered four elements. 
First, to ensure that the demonstration would begin quickly, MDRC sought 
projects that had already successfully operated employment-related programs 
for mentally retarded or other handicapped Individuals. Second, to ensure 
that each project could reach a goal of 40 to 55 slots, MDRC considered 
only those operators who could provide a solid organizational structure and 
who had established concrete relationships with potential referral agencies 
and service providers. Third, MDRC required that ptogram operators 
demonstrate that they could generate a large portion of local operating 
costs Independently of MDRC, thereby demonstrating both their commitment to 
the demonstration and their capacity to continue operations after the end 
of the demonstration. Fourth, the communities In which the projects were 
to operate were to be large enough to ensure an adequate flow of program 
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Although rural settings were considered, they were ultimately 
ruled out due to population-size limitations. 
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applicants without significantly affecting the opportunities available to 
persons who would not participate in the demonstration. 

This last consideration — an adequate community size— has important 
implications in terms of the research effort. In order to assess the STETS 
intervention accurately, the introduction of the program into the community 
should not have altered the local service and employment environment.^ If 
the STETS program induced other programs to change the services they 
provided or the persons they served, or if STETS changed the local labor 
market, then the impacts of the program on clients would be masked by the 
effect of a changing environment. Selecting adequately sized service 
delivery areas and labor markets relative to the modest program sizes 
helped minimize the likelihood of such adverse population affects and the 
concomitant analytical bias. 

3. Client Eligibility Criteria 

MDRC established eligibility criteria for two seemingly competitive 
purposes. The first and obvious one was to limit program participation to 
those who could potentially benefit from program services. The second and 
less obvious purpose was actually to encourage projects to recruit and 
enroll a broad range of clients in order to provide an adequate information 
base for examining the suitability of STETS for a diverse population. 
Thus, rather broad eligibility criteria were necessary to meet these two 
goals. Accordingly, each client was to meet the following criteria: 

• Age between 18 and 24, inclusive. This group was 
chosen to enable the program to focus on young 
adults who were preparing for or were undergoing 
the transition from school to work or other 
activities. 

• Mental retardation in the imderate, «lld, or lower 
borderline range. 2 This criterion was indicated by 
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As we discuss in Section II. B, it is particularly important that 
the environment faced by the control group remain relatively unchanged. 

Moderate, mild, and borderline mental retardation are defined by 
IQ scores in the respective ranges of 36 to 51, 52 to 68, and 69 to 80. 
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an IQ score of between 40 and 80, or by other 
verifiable measures of retardation. Specifically 
because IQ scores have been challenged as a valid 
measure of employability, special efforts were made 
to recruit applicants whose IQs were in the lower 
ranges, to ensure that the effectiveness of the 
program for this group could adequately be tested. 

• No unsubsidized full-time eaploynent of bIx or «ore 
nonths in the two years preceding intake , and no 
unsubsidized eaployaent of wore than 10 hours per 
vedc at the ti«e of intake into the program. This 
criterion was established to limit enrollment to 
persons who would be likely to need the intensive 
employment services envisioned in the model. 

• Ho secondary disability that would make on-the-job 
training for competitive employment impractical. 

While the demonstration was designed to test the 
effectiveness of the program for a disabled 
population, it was recognized that some individuals 
would have secondary disabilities of such severity 
that these individuals could not be expected to 
work independently in a regular job. While the 
projects were required to make such determinations, 
they were encouraged to apply this standard only in 
exceptional cases. 



Projects were enccuraged to recruit and enroll a broad range of 



clients in three specific ways. As we just noted, they were encouraged to 
apply the standard on secondary disability only in exceptional cases. 
Essentially, projects were expected to work with clients who had secondary 
disabilities, if those disabilities did not make the training and placement 
impossible or unreasonably difficult for the client or the project. In 
addition, projects were encouraged to enroll relatively more lower- 
functioning clients than they might have otherwise. Specifically, as their 
goal, half of their total population of clients were to have IQ scores in 
the range of 40 to 60. Finally, projects were discouraged from enrolling 
only those whom they considered in advance to have the highest likelihood 
of success in the program (i.e., they were discouraged from "creaming"). 
Such individuals may also have been those who would have been more 
successful outside of the program, and a key purpose of the demonstration 
was to determine for whom the program would have the greatest effects. 



18 




B. RESEARCH ISSUES AND EXPERIMENTAL DESIGN 

As described earlier, this report focuses on the impacts of STETS 
on e/i erimental group members (i.e., program participants) and on the 
program^s benefits relative to its costs. To be judged effective, a 
transitional-employment program for mentally retarded individuals must be 
evaluated along the same dimensions as other employment and training 
programs. In addition, transitional-employment programs have important 
social objectives that must be considered. However, because the evaluation 
can be designed to address only a limited number of issues efficiently, it 
is necessary to consider previous experience and policy interests to 
determine the issues and hypotheses that both are policy-relevant and can 
be analyzed within the context of the demonstration. (Chapter I reviews 
many of the considerations that defined the research focus.) Accordingly, 
the STETS research plan was designed to address five basic questions: 

1. Does STETS improve the labor-market performance of 
its participants? How and to what extent? 

2. Does STETS affect the use of alternative programs 
by participants? Which ones? 

3. Does STETS participation help individuals lead a 
more normal life-style? 

4. In what ways do the characteristics and experiences 
of participants or of the program influence the 
effectiveness of STETS? 

5. Do the benefits of STETS exceed the costs? 

In order to formulate a research design, we translated these broad 
questions into specific hypotheses, which are discussed in Section B.3 of 
this chapter. However, this translation necessitated that we define the 
most relevant comparative alternatives to STETS and how these alternatives 
could be incorporated into the research design. Thus, before discussing 
the hjpotheses, we address issues pertaining to alternative programs and 
services. 
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1. Alternatives to STETS 

Most of the questions examined In the demonstration ask In one form 
or another whether STETS Induces participants to change their behavior from 
what It would have been In the absence of the program. To assess this 
change, one must determine the actual behavior of participants and what 
their behavior would have been had they not been offered the opportunity to 
enroll In STETS. 

Determining the actual behavior of participants Is relatively 
straightforward — It can be determined from Interviews with participants or 
their proxies. Determining the alternative situation Is more difficult. 
Individuals who entered STETS were those who were seeking to enter the 
regular labor force. Accordingly, they were In the ref erral/case- 
raanagement network, which Includes the school systems, sheltered workshops, 
departments of vocational rehabilitation, and similar agencies. If 
participants had not entered STETS, most would have continued or enrolled 
In the traditional types of programs offered by these agencies. Of course, 
some Individuals might have decided not to enter any formal program, and 
would have stayed at home or sought jobs Informally. Therefore, the 
appropriate alternative to which STETS should be compared Is the mix of 
alternative programs that would have been used by participants In the 
absence of STETS. 

The most straightforward way to estimate what the behavior of STETS 
participants vjould have been ir the absence of the demonstration is to 
observe the behavior of a control group which is identical to STETS 
participants, but which was not given the opportunity to enroll in the 
demonstration program. Control group members were to resemble the 
participants in terms of personal characteristics and pre-STETS behavior, 
and they were to have had the same opportunities that STETS participants 
would have had in the absence of the demonstration. 

Although STETS could potentially be judged relative to a 
counterfactual situation in which no other trc;atments existed, this 
alternative is neither practical nor particularly relevant. Because 
individuals who were referred to STETS were in the service delivery system, 
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they were expected to be involved in programs or treatments in the absence 
of STETS; any test of program utility or worth should thus be made relative 
to the alternative service-delivery system that would have been replaced or 
complemented by STETS» 

2. Experimental Design 

Because STETS was introduced in the sites as a new or expanded 
program, its introduction was compatible with the experimental design. The 
basic plan was to assign eligible STETS referrals randomly to either the 
experimental group (which was given the opportunity to enroll in STETS) or 
the control group (which was not offered STETS services for the fall 
duration of the study, but which could have used other services available 
in the community). The goal of this procedure was to produce two groups 
that would be virtually identical in terms of both observable 
characteristics (e.g., age, IQ, gender, and pre-STETS activities) and 
unobservable characteristics (e.g., motivation and ability). Some 
differences might still have arisen by chance, but they should be small and 
can reasonably be controlled for statistically in the course of the 
research. 

Random assignment was planned as part of the sample intake 
process. Projects were permitted to recruit and screen applicants in any 
manner that was consistent with program rules. However, because random 
assignment was judged to be so critical to the integrity of the research 
design, it was implemented in a manner whereby the evaluation contractor, 
Mathematica Policy Research, Inc. (MPR), could control the process 
carefully. Essentially, when an applicant indicated his or her willingness 
to cooperate with the requirements of the program and the research, and 
when the project determined its willingness to accept the applicant, the 
intake worker called MPR to verify that the individual was a first-time 
applicant and to receive the applicant's assignment to experimental- or 
control-groap status. The full order of assignments was generated randomly 
for each project prior to the stare of the demonstration. These lists were 
not shared with the projects but were instead maintained by MPR, which 
assigned the statuses (experimental or control) to applicants in the order 
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in which applicants were determined by projects to be appropriate for the 
STETS program. 

3. Hypotheses of Program Effects 

The impact analysis consists of four specific areas into which the 
various outcomes of interest fall logically: (1) labor-market behavior, 
(2) training and schooling, (3) public transfer and other program use, and 
(4) economic status, independence, and life-style. In addition, the 
benefit-cost analysis considers STETS from the perspective of an 
investment. 

Labor Ma rket Behavior . The primary labor-market outcomes of 
interest are employment, earnings, and hours worked. Other outcomes 
include job characteristics and job tenure. Increased employment in 
unsubsidized, competitive jobs was the primary objective of STETS: the 
demonstration was based on the hypothesis that STETS would enhance the 
ability of participants to obtain and hold jobs in the regular labor 
market. Conversely, it was hypothesized that STETS would prompt 
participants to rely less on sheltered workshops, activity centers, and 
training jobs (in the postprogram period). 

STETS should also have an impact on earnings and work hours. It 
was hypothesized that the training .and work experience of STETS would allow 
participants to work more hours and earn more money. Wage rates may also 
be affected by STETS, but the direction of the effect is less clear. 
Overall, Individuals should have been able to perforni a^re capably in jobs, 
thereby earning higher wage rates. However, some marginal workers 
(individuals who would not have been able to work in competitive jobs 
without STETS services) were also expected to be able to obtain jobs 
subsequent to their STETS experience. Therefore, for experimentals as a 
group, the higher wages of more able individuals who performed more capably 
in jobs would tend to be offset by the lower wages associated with some 
less able individuals* obtaining jobs for the first time. 
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This procedure was based on the system developed by MPR for the 
national Supported Work demonstration. 
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Also important in assessing STETS are job characteristics. Other 
than wage rates and hours, occupation is the primary characteristic that 
can be observed. As with wage rates, it was hypothesized that improvements 
in area employment rates would increase the likelihood that less able 
individuals would obtain competitive jobs for the first time. To the 
extent that this likelihood occurs and that the jobs obtained by these less 
able individuals tend to be in low-skill occu) itions, program-induced 
improvements in job quality for more able individuals may be masked. STETS 
was also hypothesized to improve the lives of its participants by improving 
their ability to hold onto and to develop in the jobs they obtained. 
Although data on continuous job-holding are not available in this study 
(for reasons cited below), we did hypothesize that, for those who held 
competitive jobs, participants would have a higher degree of job retention 
than controls — a hypothesis that can be tested using point-in-time data. 
Similarly, for those who lost a job, participants were hypothesized to be 
more likely than controls to find another. 

Training and Schoolinf^ . Central to the evaluation is the question 
of how STETS affects the use of training programs and schools, programs 
that can lead to self-sufficiency. Almost by definition, STETS 
participation should induce a short-term increase in the use of training 
programs. However, the use of training programs other than STETS should 
decrease. In the longer-run (i.e., beyond the period of STETS 
eligibility), STETS was hypothesized to reduce the use of all training 
programs as a result of increasing employment. The effect on school 
attendance was expected to be consistent over time: participants were 
hypothesized to be less likely to attend school, both during and after 
program participation. 

Public Transfer Use . A large portion of the population eligible 
for STETS was also eligible for one or another form of public transfers. 
The main program from which mentally retarded young adults may draw 
benefits is Supplemental Security Income (SSI). Others may draw benefits 
under Old Age, Survivors, and Disability Insurance, commonly referred to as 
Social Security Disability Insurance (SSDI). In general, individuals who 
are covered by SSI are also eligible for Medicaid or the state alternative. 
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and those who are covered by SSDI are eligible for Medicare* Individuals 
and their families may also be eligible for welfare — Aid to Families with 
Dependent Children (AFDC), general assistance, food stamps, or subsidized 
housing. 

The strongest hypothesis was that STETS participation would reduce 
dependence on SSI and SSDI, as well as on Medicaid and Medicare, through 
its effects on employment and earnings. However, the SSI/SSDI effects 
should have occurred with a lag, because of delays in reporting and in 
benefit adjustments and because of provisions in the regulations that 
provide some protection to recipients. A similar hypothesis was made for 
welfare and food stamp receipt. However, this hypothesis is weaker because 
entitlement under these formb of transfers is usually based on family or 
household units, and the financial well-being of a STETS participant may 
not have substantially changed the financial well-being of the family. 
Finally, the effect on subsidized housing is ambiguous: participants were 
hypothesized to become more independent in their living arrangements, but 
this could have been accomplished, for example, through a move from their 
families to some form of subsidized housing, or from subsidized housing to 
an independent living situation. 

Economic Status, Independence, and Life-Style . STETS seeks to make 
participants more self-sufficient in an effort to enable them to lead life- 
styles that are more compatible with the general population. The economic 
status of participants was expected to improve, with earnings gains being 
greater than transfer losses. Other effects were expected to occur over 
the long term as earnings gains were realized: STETS was hypothesized to 
lead to living arrangements , financial management skills , social behavior , 
and service use that would be more compatible with the general population. 

STETS as an Investment . As an investment , STETS can be viewed from 
various perspectives, including participants, nonparticipants (i.e., all 
others in society), and society as a whole. From the participant 
perspective, STETS was hypothesized to generate benefits that exceed 
costs. From the nonparticipant perspective, the hypothesized effect is 
ambiguous: program operating costs would be offset to an unknown degree by 
savings in many other areas. From the social perspective, STETS was 
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generally hypothesized to generate benefits that exceed costs, but the 
hypothesis is somewhat ambiguous. Complicating the test of these 
hypotheses is the fact that the program was likely to generate substantial 
social and psychological benefits (and some costs) that could not be 
valued, and that would be considered but not valu. ' ex->licitly in analyzing 
STETS as an investment. 

Subgroup Analysis, The preceding sections have identified the 
basic hypotheses for the economic and social outcomes of interest. In 
addition to testing these basic hypotheses, it is also useful to examine 
the extent to which the effects of the program differed among sample 
subgroups and program features, since information on these differences will 
be helpful in planning and targeting future transitional-employment 
programs. With exceptions, the hypotheses pertaining to differences among 
subgroups are ambiguous. They are also not necessarily consisteat across 
outcome variables. Therefore, in this section, we present a brief overview 
of the subgroups of interest rather than speculate on the specific 
hypotheses. 

The subgroups that are considered fall into the following 
categories : 

• Demographic characteristics 

• Personal characteristics 

• Previous experience and attainment 

• Program features and characteristics 

The first category, demographic characteristics , includes such 
factors as age, race or ethnicity, and gender. Previous studies of 
disadvantaged groups (although typically not mentally retarded individuals) 
often show that program effectiveness can vary along such dimensions. Such 
variation can be caused by many factors, including actual social or 
cultural differences among people or differences perceived by potential 
employers • 
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The second category, personal characteristics , pertains to more 
Individual-specific traits > such as Intellectual ability, causes of 
retardation, and handicaps other than retardation. Intellectual ability Is 
measured In this study by IQ scores, and the broad range of scores 
permitted by the eligibility criteria suggest that It may be possible to 
Identify reasonably distinct subgroups. Documenting the other two 
characteristics is more difficult \d.thin the context of the research data 
collection effort, and the subgroups will undoubtedly be distinguished 
imperfectly. In fact, since the eligibility criteria imposed limits on the 
nature of secondary handicaps, the variation in terms of this 
characteristic is likely to be limited. 

The third category, the activities and experience of sample members 
in the period before they enrolled in the program, constitute a collection 
of factors that reflect both the obvious direct experiences and whatever 
personal characteristics cannot be observed directly. These variables 
Include primarily baseline or pre-baseline measures of the various outcome 
variables. 

While the fourth category, program features and characteristics > 
would appear to include obvious candidates that would condition program 
effectiveness, it presents two serious problems. In terms of the variables 
that describe the programs themselves, the distinctions among them cannot 
easily be quantified, nor do only five projects provide enough variation to 
distinguish among program features. Therefore, site is the only variable 
of this type that can be incorporated in the analysis. Of course, any 
program effects that are associated with site might also reflect 
differences among the local areas (i.e., in terms of job opportunities, 
alternative services, etc.). In terms of the variables that reflect the 
specific services provided to individual STETS participants, the fact chat 
rervlces werf* assigned at least to some degree on the basis of need rathar 
than on the basis of random selection introduces a selectivity problem into 
the analysis that might produce biased estimates of subgroup effects. 
Despite this problem, we selectively consider a few key subgroup 
classif icatio:., of this type in analyzing the impacts of the program on 
earnings, taking precautions as best we can to ralninize selectivity bias in 
the Impact estimates. 
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C. DATA COLLECTION 

Testing the various research hypotheses requires data on the 
activities and experience of expert sencals and controls from the time they 
first came into contact with the prograia and were randomized into the 
research sample to a point 22 months later. Thus, the requisite data 
include information on labor-market activities, participation in training 
and schooling programs, the receipt of transfer payments, the use of 
support services, and other activities pertaining to self-sufficiency, as 
well as information on important demographic and personal 
characteristics. These data had to be collected in a standardized manner 
for all sample members (whether in the experimental or control group), over 
time at inter"als that would appropriately capture the effects of the 
intervention ar.d key program events, and consistently across the five 
demonstration sites. 

Evaluations of many employment programs have relied on self- 
reported data collected in interviews. The difficulties experienced by 
STETS sample members in terms of physical and cognitive functioning and 
communication skills raised serious concerns about the quality of self- 
reported data. However, alternative sources of data— primarily proxy 
respondents (such as parents, guardians, or counselors) and administrative 
records from service agencies or public assistance programs—have their own 
limitations in terms of comprehensiveness and accuracy. After a careful 
consideration of the issues, we developed a data collection strategy that 
relied on multiple sources, with the primary source Heing self-reports from 
sample memb.rs. However, these data were supplemented by data from proxy 
respondents that were collected when sample member dat- were missing or 
inconsistent and by records data that were collected from referral and 
other agencies and the STETS projects, which provided critical baseline 
measures and details on eervice utilization. 

The remainder of this section briefly discusses the data collection 
design, and it reviews several important methodological and fielding 
issues. Because of the importance and complexity of the data collection 
issues, the topic is considered in more detail in Appendix B. 
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The data required for the STETS evaluation were to be collected 
through an integrated system chat included the following: 



• Interviews with the sample members and^ as necessary ^ with 
proxy respondents 

• Corroborating information provided by community service 
agencies with which tha sample members had contact and 
that were mentioned during the interviews 

• Background information on sample members collected by 
STETS project stpff as part of the intake process 

• STETS program participation data on all experimental group 
members 

• Information on program costs collected from the 
demonstration accounting systems 

• Observations of the STETS work activities of a sample of 
experimental group members 



As noted above, in-person interviews that were conducted with the 



sample members provided the majority of data for the STETS evaluation. The 
survey design that was adopted reflects a compromise between (1) obtaining 
data at key points in time relative to the receipt of program services and 
maximizing the length of the follow-up period, and (2) conducting the 
evaluation within a fixed budget. Under this design, interviews were 
scheduled to be administered at several key points in time relative to 
program application: 



• Immediately after random assignment to the experimental or 
control group (the baseline interview) 

• At a point when the majority of experimental group members 
were still actively participating in the STETS project 
(the 6-Tnonth interview, conducted approximately 6 months 
after an individual's random assignment) 

• At a point when most experimental group members were no 
longer receiving STETS services (the IS-raonth interview) 



!• The Sample Member Survey 




• At a point well beyond the end of the demonstration 
program services for all experimental group members (the 
22-month interview)^ 



All participant and control group members were scheduled to receive the 
baseline, 15-, and 22-raonth surveys. However, due to resource constraints, 
only a randomly selected two-thirds of the full sample were scheduled to 
receive the 6-month interview, which provides data primarily for estimating 
in-program effects. 

All interviews collected point-in-time data on employment, job 
training, and schooling; on involvement in life-skills training, 
recreational activities, counseling, and transportation assistant 
programs; on the receipt of cash and in-kind transfers; and on living 
arrangements and other measures of independence. These interviews weve 
designed to collect quality information relative to the time of each 
interview (point-in-time data) because recall problems for many mentally 
retarded respondents precluded obt .ining "time-line" data, as are often 
collected for studies of this type. Point-in-time data are less than ideal 
for the impact analysis, and they necessitate conducting some extrapolation 
between points-in-tiiue for purposes of the benefit-cost analysis; 
nonetheless, they do enable us to obtain unbiased estimates of treatment 
effects at critical points in time. 

2. The Proxy Survey 

When the completed sample member interview contained missing or 
inconsistent data on specified, critical items, proxy respondents were 
identified and interviewed, with the written consent of the primary 
respondent. The proxy respondent was selected in the following order of 
priority: 



1. A live-in parent or relative who provided help with 
financial management 
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Although an initial goal had been to collect three years of 
tollow-up data, funding limitations precluded such an effort. 
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2. Any other person who provided help with financial 
management 

3. A live-in parent or relative who provided no help with 
financial management 

4. A social worker or caseworker 

5. Someone whom the sample member indicated was generally 
knowledgeable 

The interview that was designed for proxy respondents was identical in 
content to the interview designed for the sample member. 

3. Service Agency Data 

Community service providers, employers, and residential service 
agencies were identified during the interviews with primary and proxy 
respondents. Each organization that was not known in advance to be a 
private employer was contacted to determine whether it was in fact a 
service agency, and to collect general information on the nature and mix of 
services to corroborate the reports of sample members on the services they 
received. The goal was to collect general data from these organizations, 
and not to identify STETS sample members specifically or to attempt to 
collect individuai-level service data. 

4. Application/Enrollment Data 

To provide a common set of data for all sample members for a point 
prior to random assignment, an application/enrollment (A/E) form was 
developed for project screening and intake staff. The forms summarized 
information from a variety of sources, including the applicant, parents or 
guardians, referral agencies, and other agencies. The form requested 
information on basic demographic and personal characteristics, as well as 
on previous experience pertaining directly to the primary outcomes of 
interest in the impact analysis. In part, the A/E form collected data that 
could not be obtained in the baseline interview, which was administered 
after the experimental group members had become involved in the program, 
when perceptions of preprogram behavior may have been distorted. 
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5. STETS Participation Data 

MDRC maintained a Management Information System (MIS) for the STETS 
projects. Information on each experimental group member was provided on a 
monthly basis to the MIS database during the period of his or her 
participation In the STETS demonstration. This information Included the 
individual's current status in the project, placement data on training and 
permanent jobs, reasons for changes in program status, the number of days 
actively involved in STETS, and the hours scheduled and actually spent on 
training jobs and in other demonstration activities. 

6. Cost Data 

To monitor the projects and to provide cost data for the analysis, 
MDRC required that the projects account for their demonstration 
expenditures. Projects w-re to record their expenditures for program 
management, training, and other services to clients. They were also 
required to maintain records of payments to clients while they were in 
Phase 1 or Phase 2 activities, including wage payments made directly by 
employers to clients. These data were reported monthly on standard forms 
to MDRC during the course of the demonstration. 

7. Work Activity Observations 

As called for in the data collection plan, observations of the work 
activities of a subsample of 40 randomly selected participants were 
conducted. These observations were designed to collect Information on the 
value of output produced by participants in their Phase 1 and Phase 2 
jobs. For each participant studied, we Interviewed the direct work 
supervisor in order to identify the following: 

• The output produced by the participant 

• The wages paid to the participant by the employer 

• The cost that would have been incurred by the employer 
to produce the output had the participant not been 
hired 
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• The additional supervisory and production costs 
incurred to produce the output due to the on-the-job 
training nature of the participant's job 



This information was subsequently integrated into the benefit-cost analysis 
to value the average in-program output per experimental, a program benefit 
that has proved to be important in previous transitional-employment 
programs. 
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III. IMPLEMENTATION OF THE DEMONSTRATION 
AND RESEARCH PLANS 



In Chapter II, we described the demonstration and research plans up 
to the point at which they were actually implemented. This chapter pro- 
vides background information on the environment in which those plans were 
implemented, as well as on the outcomes of both the sample selection and 
data collection plans. We begin by providing brief descriptions of the 
local projects selected by MDRC to operate STETS programs under its 
direction. In Section B, we present a more detailed discussion of the 
sample selection process—specifically, recruitment, sample size, and the 
implications of sample size for the research plan— and of the 
characteristics of those who were selected to participate in the STETS 
demonstration. In Section C, we discuss the response rates to the surveys 
and the quality of our data. We conclude the chapter by providing a brief 
review of the analytic methodologies that will help form the foundation for 
the various impact analyses and the benefit-cost analysis which follow in 
the remainder of the report. 

A. THE PROGRAM ENVIRONMENT 

Operators in five cities—Cincinnati, Ohio; Los Angeles, 
California; New York, New York; St. Paul, Minnesota; and Tucson, Arizona- 
were selected to implement STETS programs. The brief descriptions that 
follow help highlight one of the project-selection criteria discussed in 
Chapter II — namely, a diversity in terms of project capabilities.^ 

!• The STETS Project Organizations 

STAR Center, an acronym for Services, Training, and Rehabilitation, 
is the largest sheltered workshop in Cincinnati, Ohio. It is a division of 
the Workshops for Retarded Citizens, Inc., which is also the parent organ- 
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ization of a job-placement service for the disabled, the Joy Center. STAR 
provides vocational evaluation, skills and job-readiness training, and 
sheltered employment for handicapped persons, with a focus on mentally re- 
tarded and emotionally disturbed individuals. In recent years, STAR has 
participated in a Projects With Industry program that offers training 
positions in Cincinnati's largest hospital. Although STAR had previously 
attempted to prepare its clients for unsheltered positions, ita efforts to 
achieve this goal increased considerably under the STETS program. 

ADEPT , an acronym for Assisting the Disabled with Employment, 
Placement, and Training, operated the STETS project in Los Angeles, 
California, in cooperation with the California Institute on numan Services 
(CIHS). ADEPT is a nonprofit agency that offers a variety of employment 
services to mentally retarded workers, although most of its efforts prior 
to STETS had been devoted to finding job pl^^cements for physically 
handicapped persons. CIHS exercised administrative control over the Los 
Angeles STETS project, while ADEPT provided the direct services to 
participants. The STETS program in Los Angeles operated in two separate 
ADEPT offices; the main office was located in Panorama City in the southern 
part of the San Fernando Valley, and a satellite office was housed in an 
ADEPT branch in downtown Los Angeles. 

Job Path , in New York City, was the only local operator with prior 
experience in transitional-employment programs for a mentally retarded 
population. While both Job Path and STETS used similar techniques, some 
distinctions are worth noting. Job Path serves mentally retarded persons 
of all ages (the average age of participants was 28 years during the imple- 
mentation phase of the STETS demonstration); STETS was targeted toward 18- 
to 24-year-oluS, and thus effected an increased emphasis on youth in the 
population served by the organization. Job Path also differed from STETS 
in terms of the length of participation in each program phase; specifi- 
cally, STETS was designed to provide a quicker transition to employment. 

Minnesota Diversified Industries (MDI), in St. Paul, Minnesota, is 
a private, nonprofit corporation that describes itself as an "affirmative 
industry." While its facility resembles a sheltered workshop in some 
respects, MDI appears to emphasize more sophisticated production techniques 
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than are typically found in workshops. It also has a more varied work- 
force than is typical of workshops: most, but not all, of its workers are 
handicapped , and mental retardation is only one of the handicaps repre- 
sented. Before its association with STETS, MDI had experimented with a few 
smaller-scale efforts to place mentally retarded workers into unsubsidized 
jobs with local firms and agencies. 

The Department of Economic Security, Division of Developmental 
Disabilities (DDD), in Tucson, Arizona, was the only government agency to 
operate a STETS project. Tne agency offers a variety of residential, 
vocational, and support services to developmentally handicapped individuals 
of all ages. Many of its clients receive employment and training in a 
range of settings, including work activity centers, sheltered workshops, 
and, increasingly, competitive employment. DDD does not Itself offer 
training services, but typically coordinates services provided by other 
organizations and refers clients to them. Developmentally handicapped 
individuals are assigned to a DDD "case manager," who prepares an 
individual development plan that specifies the client's needs and goals and 
defines the strategies for achieving them. In STETS, DDD assumed more 
direct responsibility for providing employment and training services to its 
cli ents. 

2. Local Service Environment 

A comparison of the service environment oi* the sites in which the 
demonstration was implemented with the service environment of other 
possible demonstration sites provides important information for interpret- 
ing the analytical results. While it is difficult to dr:.^.' any sort of 
quantitative comparison, it would appear that each area in which STETS was 
implemented was relatively service-rich. That is, although we were not 
aware of any other large-scale transitional-employment programs that were 
operating in any of the sites, mentally retarded individuals were served by 
case-management programs, workshops and activity centers, the cchools, and 
a variety of other programs. Furthermore, all five sites were very recep- 
tive to the demonstration program, as shown at the proposal stage by their 
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commitments for referrals, local funds, and complementary services, and by 
their assessments of employment prospects. 

To the extent that the receptivity of the local environment facili- 
tated implementing the program, it may have boosted program effectiveness 
beyond what could have been expected in other settings. However, the 
potential net gains from similar transitional-employment programs would be 
greater among programs that were operated in sites that offered fewer 
alternative services to its mentally retarded citizens. The primary ob- 
stacles to achieving this greater impact potential are the greater start-up 
problems that might occur as a result of both .he relative inexperience of 
the site in serving this client group and the potentially less favorable 
attitudes of relevant community members toward increasing the level of 
employment and the degree of economic Independence of mentally retarded 
young adults. 



B. THE RESEARCH SAMPLE 

The sample of persons who were enrolled in the demonstration 
reflects the outreach, recruiting, and screening activities of the five 
demonstration projects. Thus, before describing the ' sample of individuals 
studied, we review the sample selection process. We then describe the size 
of the sample (and its implications for the research), the characteristics 
and experience of experimentals prior to enrolling in STETS, and, finally, 
the validity of the control group. 

!• Sample Recruitment and Selection 

As we indicated in Chapter II, program operators were permitted to 
develop their ovm methods for recruiting applicants into their projects; 
however, in all cases, it was essential that referrals be made from 
agencies that could verify that the applicants were mentally retarded. By 
far the most important referral sources were the vocational rehabilitation 
agencies and the school systems, but referrals also came from workshops, 
agencies for the developmental ly handicapped, and a variety of other 
governmental and private organizations. Some projects also relied on the 
news media to gain general acceptance for the program, but not necessarily 
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to attract applicants directly. Thus, virtually all of the individuals 
recruited for STETS were in the service-delivery system and were referred 
to the program by case managers. 

2. Sample Size 

New York (Job Path) was the first STETS project to accept enrollees 
on a trial basis, in October 1981. In November 1981, New York, Cincinnati 
(Star Center), and Tucson (DDD) began the intake process for inaividuals 
who became part of the research sample. These projects were joined by Los 
Angeles (ADEPT) and St. Paul (MDI) in March 1982. All projects phased out 
intake by December 1982. The final sample eligible for inclusion in the 
impact and benefit-cost analyses consisted of 437 individuals who completed 
the baseline and one or more follow-up interviews — of these, 226 were 
randomly assigned to the experimental group, and the other 211 were 
randomly assigned to the control group. In total, up to 287 of these 
sample members have been included in analyses of the 6-month impacts; up to 
415 have been included in analyses of the 15-raonth impacts; and up to 403 
have been included in analyses of the 22-month impacts. A total of 58 
program participants were excluded from the analysis — 48 v >j enrolled prior 
to random assignment, 2 who failed to complete the baseline interview, and 
8 who completed the baseline but none of the follow-up interviews. 

The final research sample size is less than half of what was 
originally planned for the study, due largely to funding constraints. This 
reduction in sample size had implications in terms of the overall strength 
of the evaluation results. In particular, the sample size affected both 
the statistical power of the impact analysis and the confidence level of 
the results. By statistical power, we mean the likelihood that the 
analysis will detect the occurrence of program effects if STETS did indeed 
have such effects; the confidence level of the results refers to the 
likelihood that we will not wrongly conclude that the program induced 
particular effects when, in fact, the program had no impact. 

Although somewhat arbitrarily, a 90 percent level of confidence and 
a 70 to 90 percent level of power are commonly adopted as minimum 
standards. By applying these standards to the STETS sample, one may 
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conclude that there is a 90 percent probability that the estimated impacts 
of the program will be statistically significant at the 90 percent level of 
confidence if the program increases job-holding, hours of employment, and 
earnings on regular jobs by, for example, 66 to 75 percent; there is a 70 
percent probability that 50 percent increases in employment outcomes will 
be detected with a 90 percent level of confidence.^ While effects of this 
order of magnitude could clearly be realized, it must be remembered that 
achieving these very large impacts is in no way a criterion for judging the 
success of the program. More modest impacts could prove to be sufficient 
to justify this type or program intervention. Thus, although the sample 
size seems adequate for estimating overall program impacts, it is important 
to consider the overall pattern of the impact estimates. 

Relatedly, it is important to recognize the substantially lower 
statistical power of the impact results for sample subgroups. For example, 
the probability of detecting program impacts as large as 50 percent of the 
control group means drops from 90 percent based on the full sample (with a 
90 percent level of confidence) to 66 percent based on a subgroup which 
consists only of one-half of the sample, and to 52 percent for a subgroup 
which consists only of one-third of the sample. Thus, especially in our 
interpretation of the subgroup results, it is important to recognize the 
severe sample size constraints and to rely heavily on informed judgments 
based on the patterns and sizes of the estimated impacts. 

While the minimum detectable differences that were calculated for 

the stated levels of precision are large relative to the expected levels of 

activities in the absence of STETS, we must remember I'hat these base levels 

of activities (i.e., the levels found among the control group) are quite 

2 

limited, and we have every reason to expect large experimental effects. 
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Had we achieved the initially proposed sample size, which was 



about double the size of the one used, we could have had a 99 percent 
probability of detecting effects in the range of 47 to 53 percent of the 
control group means and a 70 percent probability of detecting effects as 
small as one-third of the control-group means. 



Smaller effects are still important to the evaluation of STETS; 
however, they are measured at even lower levels of statistical precision. 
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The major area of concern pertains to the subgroup analysis. We must be 
very careful how we interpret experimental effects for subgroups, since the 
absence of a statistically significant effect may in some cases be due to 
the lack of statistical power associated with the small sample size. 
Accordingly, for some components of the analysis in which it is important 
to calculate differences based on small subgroups, the analysis must be 
regarded as largely descriptive, and we cannot apply our usual statistical 
standards. Furthermore, while we will analyze experimental effects for 
subgroups, the sample size will not support a statistical analysis of the 
differences in effects among such groups: not only are the subgroup sample 
sizes small, but the differential effects among groups are also expected to 
be relatively small. 

3. Characteristics of the Experimental Sample 

This section describes the characteristics of the experimental 
group. We then consider the program experience of the experimental group 
and a few relevant issues pertaining to the control group. 

As shown in Table III.l, the experimental sample was fairly evenly 
distributed across projects, with the exception of St. Paul: 20 percent of 
the respondents were located in Cincinnati, 22 percent in Los Angeles, 25 
percent in New York, 22 percent in Tucson, and just 12 percent in St. 
Paul. Males constituted 58 percent of the sample. The average age at 
enrollment was 20 years; 34 percent were 22 years of age or older. Over 
half of the sample members were white, 30 percent were black, and most of 
the remainder were of Hispanic origin. 

The experimental sample was concentrated more within the mild range 
of retardation than had been intended: only 12 percent of the sample were 
in the moderately retarded range, while 60 percent were in the mild range, 
and 28 percent were in the borderline irange. In the sample, 39 percent 
suffered from secondary handicaps. Referral agencies reported a specific 
organic cause of retardation for 18 percent of the experimental sample 
members. 
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TABLE III.l 



CHARACTERISTICS OF THE RESEARCH SAMPLE AT BASELINE, 
BY EXPERIMENTAL AND CONTROL GROUP STATUS 
(Percent) 



Characteristics 



Experimental 
Group 



Control 
Group 



Total 



Site 

Cincinnati 
Los Angeles 
New York 
St. Paul 
Tucson 

Age 

Younger than 20 
20 or 21 
22 or older 

Gender 

Male 
Female 

Race/Kthnlcity 

White 
Black 
Hispanic 
Other 

IQ Level 

Borderline 
Mild 

Moderate 

Secondary Handicap 

Organic Cause of Retardation 

Raised by Uorkiqg Parent( a) /Guardians 

FarenC(8) Held "Ifhite Collar" 
Job 

Any Benefactor 

Receipt of Transfers^ 

SSI or SSDI u 
Any cash transfers 
Medicare/Medicaid 
Any transfers 

independent Financial 
Managenent Skills 

Liviiw Arraqgeaent 

Living with parents . 
Living in group home" 
Living semi--independently 
Living independently 

Active in Job» Training, or 
School Duriqg 6 Months Prior 
to Referral 

Activities Duriqg Tuo Tears 
Prior to EnrollMnt (of at 
Least 3 Months^ Duration)^ 

Regular unsubsidized Job 

Any job 

Job training 

Attendiqg School When 
Referred to STETS 



19.5 
22.1 
24.8 
11.9 
21.7 



31.9 
34.5 
33.6 



57.5 
42.5 



54.0 
30.1 
13.3 
2.7 



27.9 
60.2 
11.9 

38.9 

18.4 

74.2 

35.1 

28.3 



33.2 
48.0 
31.4 
63.5 

28.8 



78.8 
6.2 
4.9 

10.2 

71.7 



14.3 
46.9 
5.3 

28.8 



20.9 
19.9 
24.2 
12.8 
22.3 



38.4 
35.1 
26.5 



57.3 
42.7 



48.8 
35.1 
14.2 
1.9 



30.3 
60.2 
9.5 

32.2 

15.2 

70.1 

31.8 

30.8 



33.5 
53.6 
35.7 
67.6 

25.2 



84.8 
6.6 
2.4 
5.7 

72.0 



13.7 
44.5 
3.8 

36.0 



20.1 
21.1 
24.5 
12.4 
22.0 



35.0 
34.8 
30.2 



57.4 
42.6 



51.5 
32.5 
13.7 
2.3 



29.1 
60.2 
10.8 

35.7 

16.8 

72.2 

33.5 

29.5 



33.3 
50.7 
33.5 
65.5 

27.1 



81.7 
6.4 
3.7 
8.0 

71.9 



14.2 
45.8 
4.6 

30.7 




Number in Sample 226 211 437 

NOTE: Cbi-*8quared and t-^tests differences in the characteristics of experimentals and controls 
revealed statistically significant differences in the percentage who were receiving cash 
transfers, the incidence of secondary disabilities* the Incidence of organic causes of 
retardation, and school attendance at referral. 

^The headings under this category are not mutually exclusive. 

*^Xncludes SSI, SSDI, waif are, and other governmental cash transfers received by or on behalf of the 
sample member. 

^Includes :2ash tram?fers (SSI, SSDI, welfare, etc.). Medicare, and Medicaid received by or on behalf 
of the sample member, and food stamps received by the sample member's household. 

^Includes living in an institutiont 
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The fact that about three-quarters of the experimentals were raised 
by at least one parent » another relative, or a legal guardian who worked 
"most of the time" provides some indication of the general socioeconomic 
background of the sample members. White-collar jobs were held by the 
persons who raised 35 percent of the sample members. Another environmental 
factor that may affect labor-market and other aspects of success is the 
presence of a benefactor, an individual who helps the sample member in a 
variety of tangible ways. At baseline, 28 percent of the sample had a 
benef actor. 

Even before enrolling in STETS, experimentals exhibited various 
levels of social and personal independence or dependence. Nearly two- 
thirds of the sample were receiving cash and/or in-kind transfers. Nearly 
half of the sample were receiving cash transfers: 33 percent were 
receiving SSI or Social Security Disability Insurance (SSDI); most of the 
remainder were receiving some form of welfare, either iid to Families with 
Dependent Children or general assistance. Thirty-one percent were 
receiving Medicare or Medicaid or other local equivalents. 

The ability to handle personal financial transactions independently 
(paying for purchases, paying bills, and banking) was demonstrated by 29 
percent of the experimental sample at baseline. There may have been 
additional sample members who were capable of handling financial matters 
independently but who had previously lacked the opportunity to do so, as 
well as others who could handle only some types of transactions in- 
dependently. 

The availability of opportunities might also have influenced 
independence in living arrangement, in that 79 percent of the sample were 
residing with parents or foster parents at baseline. Under half of the 
remainder (10 percent of the sample) were living independently; 5 percent 
of the sample were living semi -independently , such as in supervised 
apartments; and 6 percent were residing in group homes and institutions. 
(Two percent of the sample had resided in institutions at some point within 
the two years prior to baseline, with less than one percent residing in 
institutions at baseline.) At baseline, less than one percent of the 
sample men^'^ars were living with a spouse or their own children. 
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While the independence of experimental sample members at baseline 
was clearly limited, many were still involved in labor-market activities. 
According to information reported on the application/enrollment forms, 72 
percent of the sample members had been active in a job, training program, 
or school during the six months prior to their STETS referral.^ While 47 
percent of the sample had held some type of job (including volunteer work) 
for at least three months during the two years before referral, only 14 
percent had held a regular, unsubsidized job. Moreover, 5 percent of the 
sample had been enrolled in a job-training program for at least three 
months during the two-year period. At the time of their referral to STETS, 
29 percent of the sample were attending school. 

4. Program Experience of STETS Clients 

Those STETS applicants who were selected to be members of the N 

experimental group were eligible to receive program services based on the 

three-phase model described in Chapter II. The actual services provided to 

the experimental group members who are included in the research sample are 
2 

summarized below. 

Phase I and 2 Experience . Phase 1 training and assessment 
activities were offered in two different types of settings, depending on 
the local program operators: in Cincinnati, St. Paul, and Tucson, Phase 1 
participants worked primarily in sheltered settings; in Los Angeles and New 
York, they were typically placed in real work settings, usually in public 
agencies or nonprofit organizations. As shown in Table III. 2, 92 percent 
of the experimental group members participated in paid employment in Phase 
1, and those who participated averaged 400 hours of Phase 1 paid 
employment. Paid employment in Phase 1 was within the 500""hour limitation 
for approximately 75 percent of the sample members. 



1 

Program eligibility criteria excluded from the sample any 
individuals who were employed for 10 or more hours per week at the time of 
application. 

2 

For further details on the actual provision of program services to 
all participants, including differences among program sites, see Riccio and 
Price (1984), Chapters III and V, This section draws on those chapters for 
specific observations about projects, although statistics have been 
recalculated for the specific sample available for the impact analysis. 
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TABLE III. 2 



LENGTH OF PAID EMPLOYMENT AND PROGRAM PARTICIPATION OF 
EXPERIMENTAL GROUP MEMBERS, BY PHASE 



Characteristics 



Phase 1 



Phase 2 



Phases 1 and 2 
Combined 



Hours of Paid Employment i%) 

1- 250 
251-500 
501-750 
751-1,000 
Over 1,000 

Average Hours of Paid Employment^ 

Average Days of Paid Employment^ 

Total Duration of Participation^ 
1 month or less 

2- 3 montho 
4-6 months 
7-9 months 
10-12 months 
13-15 months 
Over 15 months 

Average Months of Participation^ 

Average Months of Inactive Status 

Average Months of Inactive Status, 
for Those with Inactive Time 



8.0 


33.6 


7.5 


21.7 


16.8 


12.4 




on A 


12.8 


21.2 


17.7 


18.6 


4.0 


6.6 


27.0 


0.4 


4.9 


21.7 


400.2 


513.3 


766.7 


74.0 


78.3 


129.9 


n.a. 


n.a. 


1.8 


n.a. 


n.a* 


5.3 


n.a. 


n.a. 


16.4 


n.a. 


n.a. 


16.4 


n.a. 


n.a. 


16.8 


n.a. 


n.a. 


26.1 


n.a. 


n.a. 


17.3 


n.a. 


n.a. 


10.6 


n.a. 


n.a. 


1.3 


n.a. 


n.a. 


2.9 



Number in Sample 
Paid Employment 
Hours and Duration of 
Participation 



208 
n.a. 



150 
n.a. 



209 
226 



NOTE: Data on paid employment were obtained from the Monthly Activity 
Forms in the STETS Information System; data on months of 
participation and inactive status were obtained from the Monthly 
Status Change Forms in the STETS Information System. 

a 

The sample is restricted to those with paid employment. 

b 

The duration of participation is measured as calendar months from random 
assignment through final transition or termination, including inactive 
time. Duration is not calculated separately for each phase, because 46 
percent of the participants experienced inactive time that cannot be 
apportioned to the phases. 



n.a. 



means not available, due to a substantial amount of inactive time that 
cannot be apportioned between Phase 1 and Phase 2. 



43 



Among the experimentals in the research sample, 66 percent 
participated in Phase 2 paid employment, averaging 513 hours. While all 
Phase 2 placements were in nonsheltered work settings, the STETS counselors 
had a high level of contact with the participants in most cases. During 
the first few weeks after a Phase 2 placement, counselors tended to devote 
several hours a day, several days a week to offering intensive training to 
participants. This counseling-provision contact was gradually reduced to 
several times a month. 

Total participation in paid emplo3nnent in STETS Phase 1 and Phase 2 
averaged 767 hours for members of the research sample,, over a period that 
averaged 10.6 months from the date of random assignment to the experimental 
group through either the transition to a job or an education/training 
program or termination from STETS. However, during this period, 46 percent 
of the sample were placed on "inactive" status for an average of 2.9 
months, and some experienced other periods when no paid employment was 
available to them. 

As shown in Table III. 3, notable differences occurred across sites 
in the degree to which the programs moved the STETS participants into Phase 
2 activity in particular, and all sites were relatively more successful in 
achieving this goal for males than for females. Overall, two-thirds of all 
program participants entered Phase 2 emplojnnent — 73 percent of the males 
and 57 percent of the females. However, these differences were especially 
notable in Cincinnati and Los Angeles, where only about half of the females 
held Phase 2 jobs, compared with over 80 percent of the males. 

Adding to the evidence which points to the differential success of 
the programs for males and females is the much lower incidence of longer- 
term Phase 2 job-holding among females: only 19 percent of the females. 



In Los Angeles, counselor work-site visits were made much less 
frequently in both Phases 1 and 2, averaging only once every three to four 
weeks. 
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TABLE III. 3 



PHASE 1 AND PHASE 2 ACTIVITIES 
BY GENDER AND SITE 



Site 



Percent with Phase 1 
Only 
Hales 
Females 



Cincinnati Los Angeles New York St, Paul Tucson 



Total 



Percent with Phase 1 
Hours > 500 
Males 
Females 

Percent with Phase 2 
Males 
Females 

Percent with Phase 2 
Hours > 500 
Males 
Females 



34.1 
29.0 



20.t^ 
22.6 
15. ^^ 

73.8 
81.0 
36.8 



34.1 
38.7 
23.1 



38.0 
29.0 
52.6 



8.0 
9.7 
5.3 

68.2 
81.0 
47.4 



36.0 
51.6 
10.5 



19.6 
13.3 
26.9 



33.9 
30.0 
38.5 

80.4 
86.7 
73.1 



26.8 
23.3 
30.8 



59.2 
57.1 
61.5 



40.7 
28.6 
53.8 

40.8 
42.9 
38.5 



18.5 
21.4 
15.4 



30.6 
20.8 
40.0 



30.7 
25.0 
36.2 

69.4 
79.2 
60.0 



26.5 
41.7 
12.0 



33.6 
26.9 
42.7 



25.7 
22.3 
30.2 

66.4 
73.1 
57.3 



29.2 
36.9 
18.8 



Number in Sample 
Males 
Females 



44 
31 
13 



50 
31 
19 



36 
30 
26 



27 
14 
13 



49 
24 
23 



226 
130 
96 



NOTE: These data are based on information obtained from the Monthly Activity Forms in the 
STETS Information System. 
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compared with 37 percent of the males, worked in Phase 2 jobs for more than 
500 hours (see Table III. 3). 

Table III. 4 describes the STETS activities of sample members as of 
the dates on which the point-in-time interview data were collected. As 
indicated by these data, the 6-month interview was administered when a 
large portion of ulie sample members were still actively participating in 
STETS; 68 percent were involved in Phase 1 or Phase 2 activities, and an 
additional 14 percent were in inactive status. The program activities of 
participants differed notably in the various sites at this point in time. 
At one extreme, 88 percent of the participants in Tucson were still active, 
and 73 percent of those who were active had not yet entered Phase 2 
employment. In contrast, only 57 percent of nhe participants in Los 
Angeles were still active, primarily in Phase 2 employment. 

By the 15-month interview, 18 percent of the experimental sample 
members were still active in Phase 1 or Phase 2, and 81 percent had 
terminated or had transitioned from the program. The one site at which a 
sizeable proportion of participants were still participating in Phase 1 or 
Phase 2 activities was Tucson'"-6 percent in Phase 1, and 37 percent in 
Phase 2. 

Placement and Follow-Up Outcomes . By original design, STETS 
participants were to move from Phase 2 into Phase 3 when the program no 
longer subsidized their wages. However, because many Phase 2 jobs were in 
fact unsubsidized (except in New York), the distinction between phases is 
unclear. Phase 3 generally involved a lower level of contact between 
counselors and participants, although some participants encountered 
problems with new regular employment or moving into independent living 
situations, which actually created a temporary need for increased contact. 

As shown in Table III. 5, an average of 44 percent of the STETS 
participants transitioned into an unsubsidized job, with the percentages 
ranging from a low of 30 percent among those in St. Paul to a high of 59 
percent among those in Tucson. In total, over half of the males who had 
participated in STETS transitioned directly into unsubsidized employment, 
while only a third of the women did so. Of the remaining participants, 
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TABLE 111 A 



PAID EMPLOYMENT AND PROGRAM PARTICIPATION STATUS OF 
EXPERIMENTAL GROUP MEMBERS, BY SITE 



. Site 

• Cincinnati Los Angeles New York St. Paul Tucson Total 

Status at 6-Month 
Interview Date [%) 

Active in Phase 1 31.0 6.7 13.9 50.0 72.7 33.6 

Active in Phase 2 27.6 50.0 50.0 22.2 15.2 3t^' 2 

17.2 16.7 16.7 16.7 3.'o 13!7 

Terminated/ 2^^o1 26.7 l^.^f li.i 9.1 19^5 

transitioned 

Status at 15-Month 
Interview Date (%) 

Active in Phase 1 2 A o.O 0,0 7.7 28 

Active in Phase 2 9.8 H.O 7.7 36.7 15*5 

2.^^ 2.1 0.0 0.0 O.'o 0^9 

Terminated/ Q5A 93.6 86.0 8^.6 57.1 80.8 

transitioned 

Percent with Paid 95.5 82.0 100.0 88.9 91.8 92.0 

Employment 5.n 
Phase 1 

Percent with Paid 65.9 62.0 80.^^ t,0.7 99A 66 ^ 

Employment in 
Phase 2 



Number in Sample: 
6-month Interview 
15-month interview 
Any follow-up 
interview 



29 30 
50 



36 18 
50 26 
56 27 



33 146 
*9 213 
*9 226 



NOTE: Data on paid employment in Phase 1 and Phase 2 were obtained from the Monthly Activity 
Forms in the STETS Information System; data on status at interview date were obtained 
from the Monthly Status Chjnge Forms in the STETS Information System. 
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TABLE III. 5 



PUCEMENT AND FOLLOW-UP OUTCOhES OF 
EXPERIMENTAL GROUP MEMBERS 



Site 



Cincinnati 


Los Angeles 


New York 


ot . r au i 


Tucson 


Total 


StatV9 at Temination (%) 














Employed in regular Job 


38.6 




D1 .0 


Cy mO 


mC. 


4^. 0 


Males 


^5.2 


^1.9 


63.3 


35.7 


66.7 


51.4 


Females 


23.1 


15.8 


38.5 


23.1 


52.0 


33.3 


Employed in workshop or in 


6.8 


U.O 


3.6 


U.8 


U.3 


10.2 


school or training 














Males 


3.2 


16. 1 


6.7 


7t 1 


16.7 


10.0 


Females 


15.^ 


10.5 


0.0 


23.1 


12.0 


10.4 


Other 


5^.5 


54. U 


/l /l ^ 

*f*f.6 




ifo . ^ 


46. U 


Males 


?1 .6 


^1.9 


30.0 


57.1 


16.7 


38.5 


Females 


61.5 


7"^ 7 




J^m 0 


^ fi 
u 




Nature of Regular Oobs Held 


11.8 


18.7 


3.6 


12.5 


10.3 


10.2 


by Those Teminated to 














Unsubsidized Eaployaent 














Sector of Employment {%) 














For-profit 


88.2 


50.0 


82.1 


87.5 


86.2 


79.6 


Nonprofit 


11.8 


18.7 


3.6 


12.5 


10.3 


10.2 


Public-sector 


0.0 


31,2 


U.3 


0.0 


3A 


10.2 


Percent rollover from Phase 2 


88.2 


31.3 


39.3 


100.0 


79.3 


63.3 


Average hours per week 


29.9 


32.2 


31.5 


31.5 


29.9 


30.9 


Average hourly wage rate 


$3.39 


$4.35 


$3.89 


$3.84 


$3.33 


$3.71 



Number in Sample 

Total^ 44 50 56 27 49 226 

Employed in unsubsidized Job 17 16 28 8 29 98 



NOTE: Data on final status and placements were obtained from the Monthly Status Change Forms in 
the STETS Information System. 

a 

One sample member who transitioned into unsubsidized employment had missing data on the 
characteristics of the Job. 
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10 percent of both the males and the females transitioned into a sheltered 
workshop or an educational or vocational training program, while 46 percent 
were terminated from STETS without having been placed in another position 
by STETS counselors. The most common reason cited by projects for 
noiy)ositive terminations was unsatisfactory attendance (comprising 18 
percent of the cases), followed by an inability to perform job tasks, to 
manage personal health problems, and to manage other family/personal 
problems, each of wliich accounted for about 10 percent of the 
terminations. These impediments to regular job-holding tended to manifest 
themselves early in the program; 57 percent of the nonpositive terminations 
occurred before participants had any Phase 2 paid employment. 

As shown in Table III. 5, 63 percent of the jobs for those whose 
final status was transition into an unsubsidized job were rollovers from 
Phase 2 jobs. Most of the placement jobs were with for-profit companies; 
Los Angeles was the only site that placed sizeable proportions of its 
participants (31 percent) in public-sector jobs. The jobs were 
characterized by an average of 31 hours of work per week, at an average 
starting wage of $3.71 per hour, translating into a starting pay level of 
$115 per week. Although the sites varied somewhat in terms of these 
aspects of the unsubsidized jobs held by former participants, the 
differences between males and females were minimal. 

STETS participants continued to be contacted by telephone and in 
person by STETS counselors after having been placed in unsubsidized 
employment as part of Phase 3. However, no specific data on the level or 
frequency of such maintenance contacts were collected. 

5. The Validity of the Control Methodology 

We noted earlier that random assignment was critical to the 
research design because it was to generate a control group whose behavior 
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"Rollover jobs" are regular jobs in which the employee began as a 
trainee in the STETS program. The job was said to "roll over" from 
training to regular status when all STETS subsidies were ended and when the 
level of trainer contact was reduced to no more than a maintenance level. 
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would be similar to the behavior of the experimental group had the latter 
not participated in STETS» Thus, experimental effects could be estimated 
directly from the differences in behavior between the two groups at 
specific time periods. Ideally, prior to program intervention, the average 
characteristics of the two groups would be identical, so as to enable us to 
base the analysis largely on simple comparisons between the groups. 
However, with random assignment, some probability always exists that the 
samples will differ along some dimensions by chance; the probability of 
such differences occurring is inversely related to sample size. Although 
small differences between the groups can be corrected by using reasonably 
sophisticated analytic procedures, major differences can undermine the 
analysis* 

As indicated by the data on the baseline demographic and personal 
characteristics of the control and experimental groups (which were 
presented in Table III.l), some differences in characteristics do exist. 
However, these differences usually involve only a few individuals. In 
terms of demographic characteristics, the only noteworthy difference is 
age. On average, the experimental group was distinctly older than the 
control group. In terms of personal characteristics, noteworthy 
differences are found for secondary handicaps, living arrangement, and pre- 
baseline activities. The experimental group also seemed somewhat more 
disadvantaged than the control group at baseline: secondary handicaps 
affected 39 percent of the former and only 32 percent of the latter. This 
finding is reinforced by the slightly higher percentage of experiraentals 
with an organic cause of retardation, as indicated in referral agency 
records. Experimental group members exhibited greater living independence 
than did controls: they were less likely to have resided with parents, and 
almost twice as likely to have lived semi-independently or independently 
(15 versus 8 percent). However, since the vast majority of both groups 
lived with their parents, the apparent difference, although large in 
percentage terms, involves relatively few individuals. Finally, control 
group members were much more likely than experimental group members to have 
been attending school when referred to STETS, although no differences in 
labor-market-related activities between the groups are notable. 
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The differences that emerged from the randomization process do not 
seem to be extreme and are generally consistent with a valid application of 
a control group methodology to a modest-size sample^ The differences that 
exist will be accounted for in the analysis so as not to bias the 
estimation of experimental effects^ In part, this process will require 
using the analytic methodologies described in Section III*D. 

C* DATA COLLECTION OUTCOMES 

Our primary concern in the data collection plan was our ability to 
obtain complete and accurate interview data from primary respondents (i»e», 
sample members) or their proxy respondents • In fact, the interview 
response rate was very high, and the data appear to be complete and of high 
quality. Here, we briefly review the outcome of our data collection plan; 
we present more details on the results in Appendix B, 

!• Interview Response Rates 

We attempted to administer a baseline interview to all 467 
individuals who were enrolled in the research sample; we attempted to 
conduct 15- and 22-month follow-up interviews with all sample members who 
completed the baseline interview, and we attempted to conduct a 6-month 
follow-up survey with only a randomly assigned two-thirds of the sample ^ 
Table III. 6 provides the results of the baseline and follow-up surveys with 
the primary sample members. 

At each wave, the vast majority of the sample completed the 
assigned interviews, from 97 percent at baseline to 89 percent at the 22- 
month follow-up. Thus, for example, 86 percent of the original sample 



1 

We determined that a two-thirds sample was sufficiently large to 
detect program impacts on the activities of sample members at month 6, 
because the dominance of STETS activities among experimentals at that point 
would have generated large impacts among key employment-related outcomes. 
(For a given level of statistical precision, larger impacts require smaller 
sample sizes.) This decision to confine the 6-month survey to two-thirds 
of the sample conserved research funds for the crucial 15- and 22-month 
surveys. 
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TABLE III.. 6 





RESULTS OF THE BASELINE 


AND FOLLOW- 


-UP SURVEYS 






WITH THE PRIMARY 


SAMPLE MEMBERS 








Percent by Final 


Status 




Number 






Other 




Assigned 


Complete 


Refusal^ 


Noncomplete'' 


Baseline Survey 










Kyua mo nt* 1 c 




99.1 


0.0 


0.9 


Controls 


231 


95.7 


2.2 


2.2 


Total 


467 


97.4 


1.1 


1.5 


6-Month Survey 










ExDG rimcntals 




94.2 


1.9 


3.9 


Controls 


148 


95.9 


1.4 


2.7 


Total 


303 


95.0 


1.6 


3.4 


15-Month Survey 










Experlmentals 


234 


91.0 


3.0 


6.0 


Controls 


221 


91.4 


2.7 


5.9 


Total 


455 


91.2 


2.9 


5.9 


22-4fonth Survey 










Experlmentals 


234 


89.3 


3.0 


7.7 


Controls 


221 


87.8 


2.7 


9.5 


Total 


455 


88.6 


2.9 


8.6 



These are refusals either by the primary sample members themselves or by 
their parents or guardians. 



These include those sample members who could not be located, who were 
deceased or incarcerated, or who had moved beyond a 50-raile radius of the 
study site. 
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completed both the baseline interview and the 22-month follow-up (8,9 
percent of 97 percent). In general, the completion rates of the 
experimental and control groups were quite similar, with the differences 
ranging from a 3 percentage point difference at baseline to a 2 percentage 
point difference at 22 months. Refusals accounted only for less than 3 
percent of the interview assignments. Although control group members or 
their parents were more likely to have refused to be interviewed at 
baseline^ probably because of their disappointment at being excluded from 
the STETS program, a slightly higher percentage of the experimental group 
refused to be interviewed in each of the follow-up waves. These overall 
results — high completion rates, low refusal rates, and small differences 
between the experimental and control groups — indicate the success of 
interviewing the primary sample. 

Refusals , Despite our success, interview nonresponse can pose 
serious problems for the analysis. Sample members may have refused to 
cooperate with the survey for a number of reasons, some of which might be 
correlated with the treatment effects. For example, experimental group 
members who had unfavorable program experiences may have wished to sever 
any further contact with the demonstration, including the survey. Thus, 
even though completion rates were very high, we investigated the likelihood 
of nonresponse bias to determine whether it posed a threat to the 
analysis. 

In an attempt to determine the sample member's employment: status, 
we conducted a special mini-survey of individuals (or their caregivers) who 
refused to respond to the 22-month survey. From our contacts with 11 of 
the 13 refusals, we determined that 5 were employed and that 4 were 
unemployed; we could not determine the status of the final two. Further, 
each status (employed, unemployed, and status unknown) was equally 
prevalent among experimentals and controls. On this basis, we concluded 
that refusals were nearly random, and would not affect the analysis. 



1 

Because each of the other reasons for interview noncompletion 
(e.g., moved out of area, unable to locate, deceased, etc.) accounted for 
very few sample members in each group, they would not likely cause 
problems. 
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Proxy Respondents » Another aspect of interview nonresponse 
pertains to using proxy respondents. We attempted to administer interviews 
to proxy respondents only when data on prespecif ied , critical items in the 
primary interview were missing or inconsistent. Based on a pilot survey 
conducted as part of the design activities in which we identified and 
interviewed proxy respondents for each sample member, we expected that 
approximately 30 percent of all sample member interviews would contain 
sufficient data problems to warrant administering an interview with a 
proxy. The actual survey outcomes, prese' ted in Table III. 7, show that, 
other than in the pilot survey (when proxy respondent interviews were 
always attempted), a proxy interview was administered less often than 
expected — in general, for less than 20 percent of the completed sample 
member interviews. In all but three instances, sample members identified a 
proxy and gave permission for an interview when one was deemed to be 
necessary, and the proxy interview was completed in virtually all cases. 
In addition to these high completion rates, proxy-interview response rates 
differ little between the experimental and control groups. 

Not all of the data from the proxy respondent interviews were used 
to construct the data files for the analyses described in this report. The 
following rules were developed to determine when the proxy respondent would 
provide information for the primary respondents: 



• The proxy data were substituted for the entire interview 
when the interviewers noted that the answers of the 
primary respondent were "very unreliable" or were 
"reliable on only some items," and that his or her 
s peech was **complet ely or s everely impaired , " and when 
the interviewers identified a consistent problem with 
the interview that prompted them to decide to contact a 
proxy. 

• The employment, training, and schooling module was 
replaced by proxy data if two or more flagged items were 
missing or if the primary respondent had been unable to 
answer one of the questions on basic activities (e.g., 
"Do you work or have a job now?"). 

• The entire transfer benefits module was replaced if the 
primary respondent reported receiving one or more types 
of cash benefits but no amount was given. 
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TABLE I II. 7 



RESULTS OF THE BASELINE AND FOLLOW-UP SURVEYS 
WITH THE PROXY SAMPLE MEMBERS 



Baseline Survey 

Experimentals 

Controls 

Total 



Percent of Primary Respondent 
Interviews Identified As 
Needing A Proxy Respondent 



Percent of Attempted 

Proxy Respondent 
Interviewers Completed^ 



50.9 (23.8)*^ 

50.7 (26.7)*^ 

50.8 (25. 3)*^ 



99.1 
99.1 
99.1 



6-Month Survey 

Experimentals 

Controls 

Total 



15.1 
23.2 
19.1 



100.0 
97.0 
98.2 



15-Nonth Survey 

Experimentals 

Controls 

Total 



15.5 
15.8 
15.7 



100.0 
100.0 
100.0 



22-Nonth Survey 

Experimentals 

Controls 

Total 



11.5 
15.5 
13. 



100.0 
100.0 
100.0 



Number in Sample: 

Baseline fy55 229 

Month 6 288 55 

Month 15 fy^5 65 

Month 22 65 



^These figures represent the percentage of the sample who were identified as needing a proxy 
interview and for whom consent was obtained for an interview with an identified proxy. At 
baseline, proxies were not identified or consent was not obtained for 3 sample members. 

^The baseline pilot sample, in which a proxy-respondent interview was attempted for all sample 
members, is excluded from the numbers in parentheses; the total sample size, excluding the 
pilot sample, i% 297. 
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• Individual critical items (such as occupation, hours 
worked, and earnings) were replaced when data were 
missing or inconsistent in the primary respondent 
intervi ew. 

As shown in Table III»8, only about 6 percent of the sample 
required proxy respondents to provide all interview ltems» Depending upon 
the interview wave, from 1 to 16 percent of the sample required proxy data 
on items pertaining to earnings or the amount of transfer payment s» For 
these items, a substantial decline in the u&x= of proxy data occurred in 
successive interview waves • 

2. Data Quality 

Because of the potential reporting problems associated with the 
primary respondents, we carefully checked the quality of the data whenever 
possible. In the initial stage of the baseline survey effort, we conducted 
a pilot study in an attempt to collect parallel data sets from sample 
memoers, proxy respondents, and agency records* This pilot study 
(Bloomenthal et al., 1982) confirmed the ability of most of the STETS 
sample to respond to research interviews and generally to provide complete 
and accurate data on themselves* Records and proxy respondents were not 
superior sources in terms of either completeness or data quality* Our 
decision was to rely on self-reports for most of the evaluation data, 
particularly for the follow-up data* As we noted, we developed explicit 
procedures to identify sample members who might tend to provide inaccurate 
or incomplete answers to interview questions, and to identify and interview 
proxy respondents in those cases. 

Throughout the baseline and follow-up data collection process, we 
collected some data from program records and through service agency 
interviews that could be used to corroborate some key components of the 
interview data. A comparison of data from the main surveys and from these 
other source£i revealed few problems with the survey data. For example, in 
comparing the survey data with the data collected from interviews 
administered to service agency staff, we found that, in all survey waves, 
fewer than 20 survey responses of any type were inaccurate. Thus, all 
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TABLE III. 8 



USE OF PROXY RESPONDENT DATA IN ANALYSIS 
(Percent) 





Baseline 


6— MonfVi 

V/ JL iV/ 11 U ti 

Follow-up 


1 S— Mnnfh 

Follow-up 


nonLn 
Follow-up 


Proxy Data Used For: 










Entire Interview 


5,9 


6.6 


5.5 


5.7 


Entire Employasent, 
Training, and 
Schooling Module 


0*4 


0.0 


0.0 


0.5 


Entire Transfer Benefit 
Module 


15.6 


9.0 


7.7 


6.7 


ProiKy Data Used For 
Individual I teas: 










Occupation 


0.2 


0.0 


0.0 


0.0 


Hours Worked 




0.4 


0.2 


0.2 


Earnings 


4.6 


3.5 


2.7 


1.2 



Number in Sample 455 288 415 403 
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available evidence suggests that the survey data on which the analysis is 
based are of very high quality. 



D. ANALYTIC METHODOLOGIES 

As we described previously, this report focuses on three issues: 
whether STETS met its employment-related and other specified objectives, 
whether and which characteristics of its participants or of the local 
projects affected its success, and whether the effects of STETS were 
sufficiently large to justify its costs. The primary outcomes of interest 
and their associated hypotheses have also been described. We now discuss 
the analytic methodologies that address these outcomes and hypotheses. 

1. Impact Analysis 

As determined by the research design (i.e., the random assignment 
of individuals to experimental or control status), the experimental and 
control groups exhibited very similar pre -assignment characteristics; thus, 
STETS participation should generally be uncorrelated with these 
characteristics. However, we did observe a few differences among these 
pre-assignment characteristics. Therefore, calculating experimental- 
control differences simply in terms of the mean values of outcome variables 
may produce biased estimates of experimental effects for at least some 
outcome variables of interest. 

Regression techniques are advantageous because they control 
statistically for sample differences of this type, and can be expected to 
produce unbiased estimates of experimental effects.^ They also offer two 
other advantages over a simple comparison of mean values. First, 
regression analysis provides more powerful tests of the potential effects 
of STETS, because we can control statistically for the influences of other 



1 

Because actual participation in STETS was nearly universal among 
members of the experimental group, the analysis is not diluted by the 
inclusion of a high percentage of nonparticipants in the experimental 
group. Further, th€» , relatively small number of nonparticipants (8 percent) 
does not warrant special techniques to isolate these individuals and their 
control-group counterparts. 
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explanatory variables. Second, by including the explanatory variables in 
the regression model, we can directly assess their individual net 
influences on outcome variables within a simple analytical framework. 
Further, by interacting an experimental status variable with other 
explanatory variables, we can attempt to isolate the effects of STETS for 
various subgroups. Thus, through regression techniques, we can begin to 
address the questions of how, for whom, and under what conditions STETS 
benefits its participants. 

What must underlie a regression analysis is a behavioral model that 
associates the outcomes of interest with a set of explanatory variables, 
some of which are predetermined or exogenous, but some of which may be 
simultaneously determined. However, a behavioral model cannot easily be 
defined for this study. For example, it is unclear how parents, guardians, 
advisors, and the mentally retarded sample members themselves interact to 
make decisions about such elements as job-holding, public assistance, or 
place of residence. If the sample members generally make their own 
decisions, it is uncertain how they perceive and react to traditional 
economic incentives. If other persons make or strongly influence the 
decisions, it is unclear who the appropriate persons are, or what their 
incentives may be. In either case, social and psychological factors may 
play a role that is potentially as great or greater than the role of 
economic factors. 

Because of the uncertain nature of the behavioral relationships and 
the possible simultaneous determination of outcomes, the regression 
analysis is based on reduced-form equations that associate the outcome 
variables with a vector of demographic, personal, and program background 
characteristics that are all exogenous. These include all of the variables 
that define possible subgroups of interest, other background variables, 
and, of course, an experimental status variable. Relying on reduced-form 
models provides accurate estimates of net treatment effects, but does not 
allow these effects to be disaggregated into their direct and indirect 
components. For example, STETS may influence living arrangements directly 
by improving participants' self-confidence and their ability to function 
independently, but it may also have an indirect influence through its 
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effect on postprogram earnings. In this analysis, the coefficient that is 
estimated for the experimental status variable in a living-arrangement 
equation will reflect the net effect of these direct and indirect 
influences. 

These models have also been expanded by the inclusion of a series 
of experimental status variables interacted with variables denoting 
personal characteristics. These expanded models enable us to estimate the 
importance of key subgroup characteristics in determining the impacts of 
the program and to test the statistical significance of these estimates. 
For example, we can estimate the impacts of STETS for males and for females 
by assuming either that these two groups are similar in all other respects 
(the option we have chosen) or, alternatively, tnat they resemble the STETS 
samples of males and females in all respects. Our rationale for opting to 
report the ceteris paribus estimates for sample subgroups is that a 
comparison of the estimates for subgroups that are thus defined provides 
evidence on the overall importance of the attribute in determining the 
impacts of the program. However, as noted in our discussion on the sample 
design, the statistical power of these subgroup results is low, and the 
power of differential impact estimates is even lower. Therefore, our 
discussion of subgroup results focuses on patterns rather than on 
statistical significance. 

The bulk of our analysis relies on ordinary least squares (OLS) 
regressions, with the various outcome variables analyzed separately for the 
6-, 15-, and 22-month data. For key binary outcome variables, we verified 
the robustness of the estimated experimental effects by using more 
appropriate probit maximum likelihood equations, as are presented in 
Appendix A. However, the OLS estimates are featured for simplicity and 
consistency. 

By necessity, we occasionally estimated experimental-control 
differences based on small subgroups of the research sample defined by one 
of the outcome measures. In such cases, the analysis is necessarily 
imprecise (as we will discuss in the next section), and the advantage of 
regression analysis is minimal. Hence, we calculated these experimental- 
control differences on the basis of simple differences of mean values, and 
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we caution the reader to view these results as descriptive data to 
facilitate interpreting the overall impact results, rather than as, 
themselves, being suggestive of the program impacts, 

2. Benefit-Cost Analysis 

A key component of the evaluation is a benefit-cost analysis that 
provides a structured method for assessing whether the impacts generated by 
the program are sufficiently large to justify the costs. The analysis 
focuses on several estimates of net present value per participant — that is, 
the difference between average benefits and average costs. The term 
present value means that the values of benefits and costs that accrue in 
the future have been adjusted to reflect their value in the present. 
Benefits and costs are computed on a per-participant basis to control 
partially for program size and to facilitate drawing cross-program 
comparisons. 

If all the benefits and costs were measured, the hypothesis test 
would be whether the net present value per participant exceeded zero. A 
positive net present value would indicate that the program represented a 
good use of resources, while a negative value would suggest that the 
resources could have been used more productively elsewhere in the 
economy. Of course, all benefits and costs cannot be measured, and many of 
those that are measured are measured imperfectly. Consequently, it is 
necessary to look beyond simple benefit-cost estimates to examine the 
relative uncertainties associated with the various estimates and the 
probable magnitudes of unmeasured ben<*fits and costs. 

The procedure adopted here is to emphasize the general patterns 
that emerge from attempting to assign relative values to effects. The 
analysis does not focus on a single net present value estimate, but ^:§ther 
on a set of estimates based on plausible assumptions and estimates. This 
set includes both a benchmark estimate that incorporates the assumptions 
and estimates with which we feel most comfortable and several alternative 
estimates based on sensitivity tests, each illustrating the effect of 
changing one of the assumptions used in the benchmark calculations (keeping 
the others unchanged). 
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Despite these procedures, a number of uncertainties remain. In 
particular, a number of program effects could not bt explicitly valued— for 
example, preferences for working, a desire to provide opportunities to 
disadvantaged persons, and personal assessments of the risks and gains of 
entering the labor market. The benefit-cost analysis uses a comprehensive 
accounting framework that includes all of these effects to enable us to 
assess qualitatively the degree to which intangible effects could alter 
conclusions based on those effects that are valued. In essence, the 
analysis provides an estimate of net cost — the value of measured costs less 
the value of measured benefits. If STETS is to be judged a desirable 
investment, the net worth of the intangibles must be sufficient to offset 
the net costs. In cases where measured benefits exceed measured costs, the 
program is usually considered desirable. In other cases, policy judgments 
must be made about whether the intangible benefits are likely to be 
sufficiently valuable to generate an overall positive net present value. 
To help render this type of judgment, our benefit-cost analysis includes a 
component which summarizes the key impacts that may be vrrelated with the 
intangible benefits and costs. 



1 

Of course, intangible costs could be large enough to offset the 
measured positive net present value. Thus, intangibles must be considered 
even when measured net present value is positive. 
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IV. IMPACTS ON EMPLOYMENT, EARNINGS, AND OTHER 
LABOR-MARKET OUTCOMES 



The overriding goal of the STETS demonstration was to identify 
effective interventions and targeting strategies for integrating mentally 
retarded young persons into the competitive labor market. Thus, a major 
objective of the impact evaluation is to measure the effectiveness of the 
program in increasing the employment levels of and improving the quality of 
the jobs held by program participants. With the experimental design 
adopted for the demonstration, the effectiveness of the program in these 
areas can be estimated reliably by comparing the employment activities and 
outcomes of the experimental (participant) group with those of the randomly 
selected control group. 

In this chapter, we first provide an overview of the expected 
labor-market experiences of the STETS participants had they not enrolled in 
the program. This overview is especially useful in providing a clear 
understanding of the nature and extent of the employment problems facing 
both this target population as a whole and selected subsets of this 
population. in Section B, we present the estimated overall impacts of the 
program on employment-related outcomes, and discuss the implications of 
these impacts in cerms of the characteristics of the jobs held by the STETS 
target population. Section C discusses differences in the estimated 
impacts across key subgroups of the target population as defined by site, 
demographic and personal characteristics, and program experiences. The 
final section summarizes the main employment-related findings from the 
demonstration and highlights the policy implications of these results. 

A. LABOR-MARKET EXPERIENCES IN THE ABSENCE OF STETS 

We can best illustrate the nature of the employment problems faced 
by the STETS target population by examining the employment experiences of 
the control group. As shown in Table IV. 1 and Figures IV. 1 and IV. 2, the 
employment prospects of these young adults will improve somewhat over 
time. To illustrate, the proportion who were employed increased from 35 to 
45 percent over the 22 months after baseline, and over 40 percent of those 
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TABLE IV. 1 

EMPLOYMENT ACTIVITIES OF CONTROL GROUP MEMBERS, BY SITE 



Site 



Activity and 




Los 


New 


St. 






Time Period 


Cincinnati 


Angeles 


York 


Paul 


Tucson 


Total 


Percent vlth Regular 




























Baseline^ 


7.0 


0.0 


3.9 


14.8 


8.7 


6.2 


Month 6 


6.9 


7A 


18.9 


0.0 


19.4 


12.0 


Month 15 


2.3 


7.7 


37.5 


14.8 


15.9 


16.3 


Month 22 


0.0 


10.5 


37.0 


18.5 


22.7 


18.6 


neroeiTC in ir«jiiing 




























Baseline^ 


20.9 


21.4 


21.6 


7.4 


6.5 


16.3 


Month 6 


20.7 


7.4 


24.3 


44.4 


9.7 


19.7 


Month 15 


9.1 


7.7 


10.4 


25.9 


15.9 


12.9 


Month 22 


2.6 


7.9 


0.0 


7.4 


18.2 


7.2 


Percent in Workshop/ 














^ivity Center 














Baseline*^ 


11.6 


11.9 


9.8 


11.1 


17.4 


12.4 


Month 6 




22.2 


13.5 


11.1 


19.4 


14.1 


Month 15 


^^A 


12.8 


14.6 


22.2 


18.2 


15.3 


Month 22 


12.8 


13.2 


15.2 


48.1 


15.9 


19.1 


Percent with Any 














Paid 3ob 














Baseline^ 


39.5 


33.3 


35.3 


33.3 


32.6 


34.9 


Month 6 


31.0 


37.0 


56.8 


55.6 


48.4 


45.8 


Month 15 


22.7 


28.2 


62.5 


63.0 


50.0 


44.6 


Month 22 


15.^f 


31.6 


52.2 


74.1 


56.8 


44.8 


Number In Sample: 














Baseline 




42 


51 


27 


46 


209 


Month 6 


29 


27 


37 


18 


31 


142 


Month 15 




39 


48 


27 


44 


202 


Month 22 


39 


38 


46 


27 


44 


194 



NOTE: These data are unadjusted subgroup means. 

Regular Jobs are those that are neither training/work-study ncr workshop/activity-center Jobs, 
''in some cases, baseline interviews were administered several weeks after random assignment. 
^Training Jobs include work -study Jobs. 
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FIGURE IV. 1 

EMPLOYMENT TRENDS AMONG CONTROLS 




mm 



Regular Job 



^121 



Month 6 



Troining Job Workshop Job 
Month 



Month 15 



Any Job 



Month 22 

122 



FIGURE IV. 2 



TRENDS IN TYPES OF JOBS HELD BY 
EMPLOYED CONTROL GROUP MEMBERS 



Baseline 



Month 6 



Regular (182) 



Training 




Training (48X) 



Workshop (35X) 




Regular (24%) 



Workshop (282) 



Month 15 



Month 22 



Training (292) 




Training (14\) 



Regular (382) 



Workshop (42%) 



Workshop (332) 




Regular (44%) 



tlOTE: This figure is based on the regression-adjusted data presented in Table IV. 4. 
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who were employed in month 22 held regular jobs (nontraining jobs and non- 
workshop/activity-center jobs) that paid an average of $3.65 per hour (see 
Table IV.2.A). Over half of these regular jobs were in the admini- 
strative-support and private-service fields (primarily messenger and food- 
service jobs), and most were full-time positions (more than 35 hours per 
week) • 

The increase in regular employment was due in part to the increase 
in the total number who were employed, and in part to the fact that some 
individuals moved from training jobs and workshop/activity centers into 
regular competitive employment jobs (see Figures IV. 1 and IV. 2) — both of 
which are trends chat are associated with the aging of the target group. 
However, thesa employment levels and trends varied both across sites and 
among subgroups de^^ined by other attributes—differences that will help 
both characterize the nature of the "successes" and highlight the nature 
and extent of the persistent problems. 

Four especially notable differences in the employment experiences 
of individuals have been found across the five demonstration sites. First, 
there is evidence that control group members in Cincinnati lost ground 
uader the economic conditions prevailing during the demonstration (see 
Table IV. 1). Second, controls in New York experienced by far the greatest 
gains in regular job employment (from 4 to 37 percent) and the greatest 
reductions in training-job employment over the 22-inonth observation 
period. Third, controls in St. Paul experienced the greatest overall gains 
in employment (from 33 to 74 percent), but over three-fourths of the jobs 
held by these individuals at the end of the demonstration period were 
workshop or activity-center jobs. Finally, the retention of regular jobs 



1 

Some portion of what we ha\ a termed ''regular jobs" may have been 
subsidized through some federal, state, or local program. However, the 
overall incidence of subsidized jobs of this nature is sufficiently small 
that, for all intents and purposes, regular jobs as we have defined them 
can generally be thought of as competitive jobs. 

2 

Appendix Table A. 11 presents area unftmplo3rraent rates in nhe 
demonstration sites throughout the time period covered by the evaluation. 
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TABLE IV.2 



CHARACTERISTICS OF JOBS HELD BY 
CONTROL GROUP MEMBERS IN MONTH 22, BY SITE 

A. REGULAR JOBS* 



Site 



Job Characteristics 


Clnd nnatl 


Los 
Angeles 


New 
York 


Stw 

Paul 


Tucson 


Total 


Percent of Saaple with 
1ledU.y Barniivs 
Less than $50 

$50 to $99 . 
$100 or nore 


n.a. 
n.a. 
n.a. 


100.0 
0.0 
0.0 


11.8 
11.8 
76.5 


20.0 
40.0 
40.0 


0«0 
50.0 

50»0 


19.4 
25.0 
55.6 


(Average weekly 
earnings) 


(n.a.) 


($32.51) 


($132.54) 


($90.07) 


($101.39) 


($106.87) 


Percent of Seaple with 
Hours Worked Per Wedc 

Less than 20 

20 to 34 

35 or Bore 


n.a. 
n.a. 
n.a. 


25.0 
25.0 
50.0 


23.5 
23.5 
52.9 


20.0 
0.0 
80.0 


20.0 
30.0 
50.0 


22.2 
22.2 
55.6 


(Average hours 
per week) 


(n.a.) 


(30.2) 


(30.2) 


(34.6) 


(34.8) 


(32.1) 


Percent of Sasple with 

Bourly Wage Eate 
Less than $3.35 
$3.35 to $4.50 
$4.51 or higher 


n.a. 
n.a. 
n.a. 


100.0 
0.0 
0.0 


41.2 
5.9 
52.9 


40.0 
60.0 
0.0 


70.0 
30.0 
0.0 


55.6 
19.4 
25.0 


(Average hourly 
wage rate) 


(n.a.) 


($1.36) 


($4.77) 


($2.82) 


($3.10) 


($3.65) 


Percent of Saaple 
with Occnpfttlon 

Sales 

Administrative, 

including clerical 
Adninis tratlve 

support^ 
Private service^ 
Fabricators^ assemblers, 

and hand working 
Production Inspectors 
Helpers » handlers, 

equipment cleaners 
Other 


n.a^ 
n.a. 

n.a. 

n.a. 
n.a. 

n.a. 
n.a. 

n.a. 


0.0 
0.0 

0.0 

25.0 
0.0 

0.0 
0.0 

75.0 


11.8 
11.8 

23.5 

52.9 
0.0 

0.0 
0.0 

0.0 


0.0 
0.0 

0.0 

60.0 
0.0 

0.0 
0.0 

40.0 


20.0 
10.0 

0.0 

40.0 
20.0 

0.0 
0.0 

10.0 


11.1 
8.3 

11.1 

47.2 
5.6 

0.0 
0.0 

16.7 


Percent with Samt 
Bsployer in Months 15 
am 22^ 


n.a. 


33.3 


75.0 


50.0 


85.7 


70.0 


Percent with Regular 
Job in Months 15 and 22!^ 


n.a. 


33.3 


87.5 


75.0 


85.7 


80.0 


Number in Regular Job 


0 


4 


17 


5 


10 


36 


Percent in Regular Job 


0.0 


10.5 


37.0 


18.5 


22.7 


18.6 



NOTE: These data are unadjusted subgroup means. 

^Regular jobs are those that are neither training/wo rk-study nor workshop/act Ivlty-center jobs. 
^Thls heading consists primarily of mail carrier and messenger jobs. 
^Thls heading consists primarily of food service jobs. 

^Thls sample includes all those who were employed in a regular job in month 15 and who completed a 22- 
Dionth interview. The sample sizes are as follows: Cincinnati^), Los Angeles-3» New York-16, St. 
Paul-4, Tuc8on-"7, and total-30. 

n.a. • not applicable 
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TABLE IV.2 



CHARACTERISTICS OF J03S HELD BY 
CONTROL GROUP MEMBERS IN MONTH 22, BY SITE 

B. ALL PAID JOBS 



Job Characteristics 



Cincinnati 



Los 
Angeles 



Site 
New 
York 



St. 
Paul 



TUcson 



Total 



Pftrc«nt of Saaple with 

IMkly Baraings 

Less than $50 100.0 

$50 to $99 0.0 

$100 or nore 0.0 

(Average weekly ($18.66) 
earnings) 

Percent of Saaple with 

Boots Votkod Pot Veek 

Less than 20 16.7 

20 to 34 66.7 

35 or nore 16.7 

(Average hours (24.5) 
per ve^) 

FOrcont of Saaple with 

Boarly Vago Kate 

Less than $0.75 50.0 

$0.75 to $3.34 50.0 

$3.35 or higher 0.0 

(Average hourly ($0.81) 
wage rate) 

Percent of Saaple 
with Occopotlon 

Sales 0.0 

Administrative, 0.0 

Includlog clerical 

Adiiinistrative 0.0 

support* . 

Private service^ 33.3 

Fabricators, asseablers, 50.0 

and hand working 

Production inspectors 0.0 

Helpers, handlers, 16.7 

equipnent cleaners 

Other 0.0 

Percent with Saae 44.4 
Eaployer In Hontho 15 

sod 22r 

Percent with Job 55.6 
Months In 15 and 22^ 



83.3 
16.7 
0.0 



33.3 
12.5 
54.2 



52.6 
26.3 
21.1 



43.5 
34.8 
21.7 



($29.65) ($103.99) ($59.57) ($60.95) 



25.0 
41.7 
33.3 

(25.9) 



50.0 
50.0 
0.0 



0.0 
0.0 

0.0 

16.7 
33.8 

8.3 
8.3 

33.3 

45.5 



45.5 



20.8 
41.7 
37.5 

(29*3) 



8.3 
50.0 
41.7 



($1.16) ($3.78) 



8.3 
12.5 

16.7 

45.8 
12.5 

0.0 
4.2 

0.0 

64.3 



75.0 



10.5 
21.1 
68.4 

(32.8) 



26.3 
52.6 
21.1 

($1.75) 



0.0 
0.0 

0.0 

26.3 
47.4 

15.8 
0.0 

10.5 

70.6 



88.2 



17.4 
52.2 
30.4 

(29o8) 



30.4 
52.2 
17.4 

($2.05) 



13.0 
4.3 

4.3 

21.7 
17.4 

8.7 
13.0 

17.4 

63.6 



95.5 



52.4 
21.4 
26.2 

($65.44) 



17.9 
41.7 
40.5 

(29.4) 



27.4 
51.2 
21.4 

($2.26) 



6.0 
4.8 

6.0 

29.8 
27.4 

7.1 
7.1 

11.9 

60.9 



77.0 



Number in Paid Job 
Percent in Paid Job 



6 

15.4 



12 
31.6 



24 
52.2 



19 
74.1 



23 
56.8 



84 
44.8 
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NOTE: These data are unadjusted subgroup means. 

*Thi8 heading consists primarily of mail carrier and messenger jobs. 
^Thia heading consists primarily of food service joDs. 

^This sample includes all those who were employed in month 15 and who completed a 22-month 
interview. The sample sizes are as follows: Cincinnati-9, Los Angeles-U, New York-28, St. Paul-17, 
TUcson-22, and total-87. 
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over the follow-up period was substantially higher in New York and Tucson 
than in the other sites (see Table IV*2,A), 

We also found some notable patterns in the incidence of job-holding 
and the types of jobs across different demographic subgroups. There 
appeared to be little variation in the probability of being employed, but 
substantial variation in the types of jobs held was evident. Table IV, 3 
shows that whites were much less likely than average to be without a job at 
month 22, as were those few persons who were living independently at 
baseline. Job-holding also varied considerably across sites; St, Paul had 
the lowest unemployment rate (26 percent) and Cincinnati had the highest 
(84 percent). However, the probability of being without a job did not vary 
substantially across the other subgroups. 

In contrast, the types of jobs held varied considerably across all 
subgroups, with the exception of the male and female subgroups. In month 
22, about 19 percent of the males and females held regular jobs; 7 percent 
held training jobs, and 20 percent held workshop or activity-center 
positions. In the absence of the intervention, the following sets of 
individuals would be more likely to have held regular jobs: individuals 
from the New York and Tucson sites, individuals who have only borderline 
retardation that does not have organic origins, individuals who are younger 
than age 22 at baseline, individuals who have some prior work experience, 
and individuals who have financial management skills. The following sets 
of individuals would be more likely to have held training jobs: 
individuals from the Tucson site, individuals who are moderately retarded, 
and individuals whose retardation has organic causes. Finally, the 
following sets of individuals would be more likely to have held a 
workshop/activity-center job: individuals from the St, Paul site, 
individuals who have mild or moderate retardation and/or retardation that 
has organic origins, and individuals who clearly exhibit dependency (as 
evidenced by their supervised living arrangement, their lack of money- 
management skills, and the presence of a benefactor). These patterns are 
consistent with prior expectations that the least handicapped individuals 
are those who were more likely to have moved into competitive employment in 
the absence of the intervention. Thus, the question to be addressed in the 
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TABLE IV ^3 



PERCENT OF CONTROL CROUP >CHeERS WITH VARIOUS EHPLOYMENT STATUSES 
IN MONTH 22, BY CHARAaERISHCS AT BASaiNE 



Subgroups Dtflned by 
Chiracf rlftlcs at Baseline 



Ewployed 



Regular 
Dob* 



Fjiployment Status 



Training 



Workshop/ 
Activity Center 



Total SauM* 

Sit* 

Cineinnati 
Lot Angeles 
New York 
St. Paul 
Tucson 

10 tMl 

Borderline 

NUd 

Moderate 

Younger than 22 
22 or older 

Qenier 

Hale 
Fenale 

Rm/Etlviftoity 
Black 

Hispanic ^ 
White and other^ 

Uvln ArffM^iaiWt 

Living with parents 

Living in supervised setting 

Living independently 

FlnaMtaL Wwninift SkllU 
Independent 
Not independent 

Heoelpt of Trawflers 

SSI/SSDI 

Other transfers only 
No transfers 

Smntfary Hantfioaps 

Secondary handicap 
No seeomlary handicap 

Cmm Ratardatloii 

Organic 
Non'^rganic 

BaMfaetvr 

Benefactor 
No benefactor 

Woiic Experlam In T«o 
Y«ars Ptior te EnrollMiit 
Regular job lasting >3 aonths 
Other .job lasting >nsonths 
Other** "~ 

SoIimX Status at Rtfarral 

Enrolled 
Not enrolled 



55.2 



m,6 

25.9 
43.2 



S3.3 
56.3 



58*9 
45.3 



55.8 



71,9 
42.6 



56.1 
33.3 
72.7 



59.6 
5?.4 



49.3 
63.2 
54.4 



53.2 
56.1 



46.7 
56.7 



42.4 

60.7 



51.9 
44.1 

62*0 



60.3 
52.1 



18.6 



0.0 
10.5 
37.0 
18.5 
22.7 



31.7 
14.6 
0.0 



20.6 
13.2 



18.5 
18.6 



14.1 
24.1 
19.8 



16.9 
16.7 
18.2 



29.6 
15.1 



13.4 
17.6 
24.6 



17.7 
16.9 



6.7 
20.7 



22.0 
17.0 



25.9 
27.1 
12.0 



12.3 
22.3 



7.2 



2.6 
7.9 
0.0 
7.4 
18.2 



6.7 
3.5 
31.6 



7.8 
5.7 



7.4 
7.0 



3.1 
3.4 
10.9 



7.9 
5.6 
0.0 



2.1 
8.f/ 



10.4 
8.8 
1.6 



12.9 
4.5 



16.7 
5.5 



6.5 
6.7 



14.8 
1.7 
8.3 



12.3 
4.1 



19.1 



12.8 
^3.2 
15.2 
48.1 
15.9 



6.3 
23.5 
26.3 



12.6 
35.8 



19.4 
18.6 



10.9 
10.3 
26.7 



17.1 
44.4 
9.1 



6.5 
22.6 



26.9 
10.3 
19.3 



16.1 
20.5 



30.0 
17.1 



27.1 
15.6 



7,4 

27.1 
17.6 



15.1 
21.5 



Nuaber in Saaple 



107 



14 



37 



NOTEt These data are unadjusted subgroup neans. These figures so«etl«es differ slightly froa regress Ion -ad justed figures 
reported In other tables. 

^Regular jobs are those that are neither train Ing/work-study nor workshop/act Ivlty-center jobs. 

^Training jobs include eork-study jobs. 

^Only 3 percent of the saaple were classfied as ''other'*,— A»erlcan Indian or Asians. 

^''Other'* includes Individuals with no prior work experience and those with less than three Months of experience during the 
two years prior to their program application. 



71 



ERIC 



123 



demonstration evaluation is whether these individuals or whether those who 
are less likely to have succeeded on their own constitute the group that 
would benefit most from the STETS treatment. 

B. OVERALL PROGRAM EFFECTS 

Most of the impact evaluation of the STETS program has been based 
on ordinary least squares (OLS) regression analysis. As noted in Chapter 
III, this technique is advantageous because it (1) controls for baseline 
differences between the experimental and control samples, (2) enables us to 
improve the efficiency of the program impact estimates over those that 
would have been generated on the basis of simple differences of means 
comparisons, and (3) facilitates investigating the impacts for a variety of 
sample subgroups. Moreover, OLS regression analysis is relatively in- 
expensive, which is an important consideration in view of the large number 
of outcome measures we are considering. However, OLS regression estimates 
of program impacts do not have desirable statistical properties when used 
with outcomes that are either truncated (i.e., which have lower and/or 
upper bounds) or binary; thus, we have re-estimated selected results by 
using appropriate maximum likelihood estimation techniques to confirm that, 
as is generally found in evaluations of this type, similar results will be 
obtained using OLS regression and maximum likelihood methods of 
estimation. 

The results of this analysis indicate that offering STETS services 
to mentally retarded young adults did have significant beneficial effects 
in terms of the incidence of competitive employment and the level of post- 
program earnings (see Table IV. 4), Furthermore, these effects tended to 
remain fairly constant between months 15 and 22, despite the fact that some 
experimentals left their program jobs during this period. 

In this section, we first discuss the estimated overall program 
impacts on employment in and earnings from various types of jobs for the 



1 

Appendix Table A.5 presents a comparison of selected regression 
and maximum likelihood estimates • 
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TABl£ IV.<^ 



ESTlmTED PnOCRAM M^AOS £MR.OV>e<T 
AaiVDY 



OutCQW Heasores 




Honth 6 






Month 15 






ykxith 22 




Experimental 
C^p Mean 


Control 
Group 


Estimated 
Impact 


Experimental 
Group Mean 


Control 
Group 
Ne«i 


Estimated 
Impact 


Experimental 
Qroup Mean 


Qroup 
He«i 


Estimated 
Impact 


ReguUr job 


11.8 


10.7 


1.1 


2S.2 


16.8 


9.4»« 


31.0 


19.1 


11.9*» 


Trainlr^ jdb^ 


^9.0 


21.8 


27.2»» 


H.O 


12.9 


1.1 


A 7 






fforlohop/aetlvlty- 




13.1 


-6.3* 


^9 


14.6 


-9.7»» 


7.0 


18.3 


-11.3** 


center 




















Any piid >}b 


^.8 


^5.2 


22.6— 


44.8 


43.6 


1.2 


44.7 


43.7 


1.0 






















ReguUr Job 


3.9 


3.0 


0.9 


7.8 


5.0 


2.8»» 


10.0 


6.1 


3.9*» 


Any piid job 


19.8 


12.2 


7.6« 


13.1 


12.7 


0.4 


13.7 


12.7 


1.0 


kmttqi MJy Evfdngs 




















Regular job 


$ 11.81 


$ 9.81 


$ 2.00 


$26.90 


$ 16.31 


$ 10.59»» 


$36.36 


$20.55 


$ 15.81»» 


^ ptid job^ 


52.39 


25.93 


26.«6»» 


37.91 


26.48 


11.43»» 


40.79 


28.41 


12.38^^ 


ti^r In Soiple 

Peroert of Experlnentals In 
Riase 1 and Rtase 2 


67.8 




283 


18.3 




402 


0.0 




395 



NOTEt These rewlts mre estimated through ordinary least square techniques. Definitions «id means of control vart*les that are included in 
the models are preserted in Appendix Table A*1. Full results from a representative set of the li^pact equatlom that underlie t»«se 
re&ilts are preserted in Appendix TAles A.2 and A.3. Results for selected outcomes and time periods were also estimated using probit 
models that account for the binary nature of the outctne measures. These results are presented in Appenllx Table A. 5. 

^legular jobs are those that are neither trainlng/worfc-study nor workshop/act Ivlty-oenter Jobs. 

'draining jobs Include wodc-study jobs. 



"In constant second quarter of 1982 dollars, these otperimental -control dlfferertlals are sli^tly smaller. 
$10.50, and $10.10 in months 6, 15, ^ 22, respectively. 

^Statistically significant at the 10 percent level, two-tailed test. 
**Statlstlcally significant at the 5 percent level, two-tailed test. 
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full sample of experimentals. We then discuss the effects of the program 
on the characteristics of the jobs held by those who are employed — effects 
that can be caused both by an induced change in the quality of jobs held by 
those who would have been employed in the absence of the program and by a 
program-induced movement of some individuals into employment. 

1. In-Program Effects 

Six months after program enrollment, when most (82 percent) of the 
experimentals were still enrolled in Phase 1 or 2 of the program and many 
(68 percent) were in program-funded training positions which paid minimum 
wages, experimentals were significantly more likely to be employed (63 
versus 45 percent), to be working more hours per week on average (20 versus 
12), and to be earning twice as much income per week ($52 versus $26). 
This increased employment among the STETS participants is attributable 
almost entirely to the program-provided training jobs. 

This influence of the STETS training jobs can be seen quite clearly 
in Tables IV.5.A and IV.5.B, which show experimental-control group 
comparisons in terms of the characteristics of the jobs held. First, 
employed experimentals were much more likely than employed controls to be 
working in administrative, administrative support, and fabrication, 
assembling, and handworking jobs — occupations that were prevalent in the 
STETS Phase I and Phase II training jobs (see Riccio and Price, 1984, p. 
43). Second, employed experimentals were more than twice as likely to be 
working full time (47 versus 22 percent). However, experimentals were 12 
percent more likely to be earning less than the minimum wage (85 versus 76 
percent). 

2. Postprogram Effects 

Under program guidelines, participants were to have spent no longer 
than 15 months in STETS. Thus, it was expected that both the 15- and 22- 



1 

Chow-tests of the acceptability of pooling data across sites 
indicated no evidence of different underlying structural models for the 
various sites. Selected results of these tests are presented in Appendix 
Table A. 6. 
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IV.5 

ESTimiH) P90CIWH W»ACrS ON 30B CHWWTOUSncS 
A. Ptm/R 3065* 



Oob Char«cterl$tics 




Honth o 






Hcnth 15 






Hinth 22 




ExperlnBrt«l 


Cbntrol 
Nam 


EstlMted 


ExperlnBrt«l 
Ooup Hotfi 


Control 
Nam 


EsvUuted 


txperlaertal 
Qraup Madn 


Oontrol 
Qroup 


EstlJMted 


Awigi Mdy Eantngt 


$ 102.x 


$ 89.70 


$ tc.€0 


$ 103.H 


$97.52 


$ 5.92 


$ 118.31 


$ 106.87 


$ 11.44 






27.2 


6.2* 


29. S 


30.0 


-0.5 


32.0 


31.9 


0.1 




$3.10 


$ 3.32 


$ <0.22 


$3.65 


$ 3.47 


$0.16 


$3.62 


$3.65 


$ -0.03 



lm%thmS50 
$50 to $99 
$100 or aore 



Less than 20 
20 to 34 
35 or mre 

taoM* or villi 

Lass thin $3.35 
S3.35 to $4.50 
$^.51 or high»r 



«lthl 

AiMnlstr«tlve» 

Including elsrlcsl 
MRinistrjtive 

•jppoit 
nrlvata asEvioe 
Fibrlottors, maeablmt 

and hand woridng 
Production Inspectors 
Hdoersy handlers, 

e(|uipiaert cleaners 
0tl«r 



In NMthi 15 ad 

In tochti IS aMB i 



12.5 
75.0 


25.0 
37.5 


-12.5 
-25.0 
37.5 


12.7 
27.3 
60.0 


18.2 
X.3 
51.5 


-5.5 
-3.0 
8.5 


11.1 
25.4 
63.5 


19.4 
25.0 
55.6 


-8.3 
0.4 
7.9 


12.5 
25.0 
62.5 


23.5 
41^ 

35.3 


e 

-11.0 
-16.2 
27.2 


2^.6 
21.0 
54.4 


18.2 
45.5 

as.3 


• 

6.4 
^4.5 
19.1 


12.5 
28.1 
S9.4 


22.2 
22.2 
55.6 


-9.7 
5.9 
3.6 


68.8 
18.7 
12.5 


50.0 
31.2 
18.8 


e 

19.6 
-12.5 
-6.3 


50.9 
X.9 
19.2 


45.5 
27.3 
24.2 


5.4 
3.6 
-6.0 


50.8 
30.2 
19.0 


55.6 
19.4 
25.0 


-4.6 
10.6 
-6.0 


0.0 
0.0 


5.9 
11.6 


e 

-5.9 
-11.8 


1.8 

7.0 


0.0 
6.1 


1.8 
0.9 


1.6 
9.4 


11.1 
6.3 


• 

-9.5 
1.1 


6.2 


23.5 


-17.3 


12.3 


15.2 


^.9 


15.6 


11.1 


4.5 


50.0 
18.8 


35.3 
5.9 


1A.7 
12.9 


S2.6 
7.0 


51.5 
3U) 


1.1 
4.0 


45.3 
14.1 


47.2 
5.6 


-1.9 
6.5 


0.0 
12.5 


5.9 
5.9 


-5.9 
6.6 


1.6 
14.0 


0.0 
18.2 


1.6 
-4.2 


1.6 
6.3 


0.0 
0.0 


1.6 
6.3 


12.5 


5.9 


6.6 


3.5 


6.0 


-2.5 


6.2 


16.7 


-10.5 


n.a. 


n.a. 


n.a. 


n.a. 


n.a. 


n.a. 


63.6 


X}.0 


-6.4 


n.a. 


n.a. 


n.a. 


n.a. 


n.a. 


n.a. 


^.5 


80.0 


-5.5 



f^JKtdt in Regular Oob^ 



33 



90 



DO 



WTEi These resilts ar« based on («i<1usted aean values and percentage dUtrlbutions for those Indlvldiials who held regular jobs at the 

referenoe period. The tesU of statistical slTdficanoe are based on t-tests or chl-stjuare tests, as w^l«te. Seen of the figures 
reported are based on slijhtly SMller sample sizes than indicated at the bottom of the tAle, due to alsslng dsta. These flgirej may 
differ sUj^tly the oorresponding daU repotted in Table IV.2, due to slls^ difftorenoe In the sanpla. This san^le U restricted to 
those cases with valid daU on the mln ai|»loy«nt-related outoows. 

^e»jlar >>bs are those that are neither training/work-study nor wortahop/activlty-center jofae. 

crtegory consists priiMrUy of mU carrier Ml Mssenger Jobs. 

^Is eatery consists prlMrlly of food service jofae. 

SiU sa^le includss all those who were ei^oycd in a reg-jlar job in month 15 and who co^>leted a 22-«onth interview. The Mople s«m «e 
as follows! scperlMental srajp-55, control ^a4)-30, «vt total-85. 

^-square statistic coild not be co^ed because o^r 20 percent of the cells have ©ipected courts of less th*i 5. 

^Becaise thU tible Includes only those who held regular jobs, the proportions of ocperiaental ^cup and control inters «e not 
approKiJwately equal, as they are in other tjbles. At months 15 and 22, the sasple coruisted of ^)prc(iiwately 63 percent nperliicrtals and 37 
percent controls, while the proportions at month 6 were nearly equal. 

*€xperimen£al-cortrol dlf ferertials In the distrUutlons/aeans are statUtlcally sis^ificant at the 10 percent level, two-taUed teat. 
♦•Experimeftal-oontrol differentials In the dUtrlbutlons/means ate statUtlcally signifloint at the 5 percent level, two-tailed test. 



n.a. « not ^licable 
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T^etE tV.5 

ESnrt^TED PWX3W4 W»AaS Oi 30B CHWCTCRISncS 
B. /llPAtOOOBS 



Kvith 6 



EstliNted 

3bb ChiMCttrUtlca 



(botrol 

Cxperl«efit«l Cstlmtad 
Qroup Mp> Mean Iwpact 



Month IS 

Oontrol 

Experl«enit«l Ctaip Estlnated 
Qrcup Hean Moan Imotdi 



Mwth ZZ 

Q>ntrol 

ExperlMeftt«l Ooup EstlMted 
Qrojp Hean Hean Inpact 



AMngi Mcly CtfiOngs 



test t)Mn$SO 
$50 to $99 
$100 or «QCO 

l^fOMt oT Sh^^Io villi 



test ttMnZO 
20 to 34 
33 or aoro 



Lflit thm $0.75 
$0.75 to $3.)^ 
$3.35 or hl^ 



«Uh Ooevpitkn 
SaLm 

ActalnUtratlvo, 

Including ckrled 
AdUnUtrttlvo 

aipport u 
Ptlvati soivloo 
Fibrlcitorir aiMi^lort, 

md hwi voddnQ 
Production Inspactrn 
HelpMrtt hwdlon^ 

eqiApiit dawn 
Othir 



villi 3Bb 



$ 74.99 
29.2 



$56.11 
26.9 



S 18.8d»» 

2.3 



S 84.33 
29.3 



S 58.x 
29.0 



S 26.0>»» 
0.3 



S ».32 
30.6 



S 64.18 
29.2 



S 26.14»» 
1.4 



$2.61 


$2.25 


$ 0.36 


$2.96 


$ .03 


$ 0.93«» 


$ 2.65 


$ 2.24 


$0.61 


23.2 
54.6 
22.2 


56.5 
27.4 
16.1 


«-» 

-33.3 
27.2 
6.1 


29.5 
28.4 
42.1 


52.9 
23.0 
24.1 


•* 

-23.4 
5.4 
18.0 


31.5 
25.0 
43.5 


53.5 
20.9 
25.6 


tt# 

-22.0 
4.1 
17.9 


t7 n 
It .1/ 

36.0 

V.0 


£ 1.7 

56.9 
21.5 


«-» 
•4.5 
-20.9 
25.5 


18.3 
36.4 
43.3 


16.9 
46.7 
34.4 


-0.6 
-6.3 
8.9 


13.6 
39.4 
46.6 


19.6 
40.7 
40.7 


-4.8 
-1.3 
6.1 


4.0 
80.8 
15.2 


22.6 
53.2 
24.2 


«-» 
•18.6 
27.6 
-9.0 


D.5 
54.7 
X7 


32.2 
47.1 
2D.7 


•* 
-21.7 
7.6 
14.0 


13.0 
52.2 
34.8 


28.2 
50.6 
21.2 


»• 

-15.2 
1.6 
13.6 


0.0 
7.1 


1.5 
4.6 


-1.5 
2.5 


1.9 
3.9 


1.1 
4.5 


0.8 

-0.6 


1.1 
6.4 


5.6 
4.6 


»* 
-4.7 
1.6 


12.1 


9.2 


2.9 


8.7 


8.0 


0.7 


11.8 


5.8 


6.0 


30.3 
23.2 


29.2 
20.0 


1.1 
3.2 


36.5 
15.4 


29.6 
14.8 


8.9 
0.6 


39.8 
15.1 


30.2 
27.9 


9.6 
-12.8 


2.0 
6.1 


6.2 
6.1 


J^.2 
0.0 


1.9 
16.3 


4.5 
13.6 


2.7 


3.2 
7.5 


7.0 
11.6 


-3.8 
-4.1 


19.2 


23.1 


-3.9 


13.5 


23.9 


-10.4 


15.1 


7.0 


8.1 


n.«. 


n.«. 


n.e. 


n.«. 


n.«. 


n.«« 


54.0 


60.9 




n.«« 




n.«« 




n.«. 


n.«. 


72.0 


77.0 


-5.0 






165 






194 






180 



fMw In PSild Oob 



NOTEt Ihooo r^t* «e based on Mdjurtod man valuet «nd peroortUdB dUtrlbutlons for those Individuals trfio held >)fae at the reftrenoe 

period. T)« tseU of statUtScal UgilfloMoe are based on t-taaU or chl-square teats, as appropriate. Sam of the fl^ra r<pocted are 
based on tll^Uy Mallar sapla slzas tim Indicated at the bottoM of the tibia, due to aUtlng data. These figures may dlf fsr sUs^Uy 
ttm the oormpondlng diU npocted In TAle IV.2 due to sU(^ dlf ftrenoes In the sasple. This amplA U restricted to those osses with 
valid daU on the aaln e^ploywnt-ralated outooaea. 

hsadlng oomlsu prlavUy of aaU oarrler and ■essenger >oba. 

^^Thls category conslsU prlMrlly of food servloe Joba. 

^Ihls SM|)le Includba all those iho were alloyed In aonth 15 and iho co^plated a 22HBonth Intervlea. The sample slzw are as follovsi 
oRperlaeftal group-lOO, control gra4>-67, and total-167. 

«Cxperlwtal-oorttrol dtfferwtlals In the dlstrlbutlonsMom «e sUtlstlcall* l^itflcant at the 10 peroerfc level, two-tailed test. 
«€xperl«ef*al-oontPol dlffbcwAlaU In the dUtrlbutlona/asans are statistically slgrafloertt at the 5 peroerfc level, two-tailed tost. 

n.a. « not sppUoiblB 
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month data would have provided measures of postprogram impacts. In fact, 
however, 18 percent of the experimentals remained in STETS for 15 months or 
longer, and thus the 15-nonth impact estimates reflect, in part, the 
direct influence of STETS employment and earnings. The 22-month results 
are truly postprogram impacts. 

As shown in Table IV. 4, the program clearly had its intended 
effects of increasing the likelihood of holding a competitive job, the 
average hours employed in regular jobs, and earnings. By month 15, 26 
percent of the experimentals (compared with 17 percent of the controls) 
held regular jobs, a 53 percent increase. A reduction in work- 
shop/activity-center jobs of roughly equal magnitude (10 percentage points) 
was also evident in this time period, while no differences occurred in the 
proportions who held non-workshop-related training jobs. 

Slightly larger but qualitatively similir results were obtained for 
month 22. The increases in ',he magnitudes of the estimated net impacts on 
regular and workshop job-boLding wers most likely due to the fact that many 
of the experimentals who were in training jobs in month 15 transitioned 
into these regular aiid workshop jobs. 

Figure IV. 3 shows quite clearly the shifts in the composition of 
the jobs held oy experimentals relative to controls. As we noted above, in 
month 6, a substantially higher percentage of experimentals than controls 
had gained employment between the baseline and 6-month interviews, and, as 
shown in Figure IV.3, almost all of this gain was in program-rellted 
training jobs. iy month 15, when the overall employment rates of experi- 
mentals and controls were similar, we began to see clear evidence that 
STETS greatly facilitated the movement of mentally retarded young adults 
out of workshop/activity-center training jobs into regular jobs, while 



10.6 months. 
2 



1 

As noted in Chapter III, the average duration of enrollment was 

n o ^ 



Given that STETS jobs were considered training jobs, the fact that 
1« percent of the experimentals were still in the program at this point 
causes a sUght downward bias in this net impact estimate and an upward 
bias in the estimated impacts on training jobs. 
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FIGURE IV.3 

TRENDS IN TYPES OF JOBS HELD BY 
EMPLOYED EXPERIMENTALS AND CONTROLS 
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NOTE: This figure is based on the regression-adjusted data presenter*, in Table IV. 4. 
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Regular (382) 
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control-group members exhibited a trend whereby they increased their over- 
all level of employment and moved out of nonworkshop training jobs into 
both workshop/activity-center positions and competitive employment. 

Accompanying these program-induced shifts from workshop /activity- 
center jobs to regular jobs was a significant gain by experimentals in 
average hours employed (from 5 to 8 hours per week in month 15, and from 6 
to 10 hours per week in month 22). Average weekly earnings, both from 
regular and all paid jobs, also increased among experimentals. by between 
$11 and $16-an earnings differential that is proportionately larger than 
the employment-rate gains. 

Tables IV. 5. A and IV. 5. B present more detailed descriptive data on 
the quality of the jobs held by experimentals and controls. In the 
postprogram period (month 22), not only were experimentals more likely than 
controls to secure regular jobs, but the regular jobs also paid average 
hourly wages that were roughly equal to the regular jobs held by controls 
($3.62 versus $3.65 per hour). When all jobs are considered, the average 
earnings gain of employed experimentals relative to employed controls was 
$26 per week, reflecting their higher incidence of holding regular jobs, 
which tended to pay substantially higher average hourly wage rates than 
paid in other types of jobs. Although a higher proportion of the employed 
experimentals held full-time jobs (47 versus 41 percent), this was not 
reflected in higher average hours of employment. 

The final dimension in which the program appears to have induced a 
change in job characteristics pertains to occupation. In terms of general 
occupational categories, experimentals and controls held quite dissimilar 
jobs during the postprogram period. Roughly 40 percent of the employed 
experimentals worked in private-service positions (primarily food service), 
and sizeable numbers were employed in administrative-support positions 
(primarily mail carrier and messenger positions) and in fabrication and 



1 

In constant second-quarter 1982 
on earnings from all jobs are $10.50 and 
are 92 and 82 percent of the nominal 
respectively. 



dollars, these estimated effects 
$10.10, respectively — figures that 
estimates for months 15 and 22, 
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assembling/handworking positions (12 and 15 percent, respectively). In 
contrast, proportionately smaller numbers of controls were employed in 
private-service and administrative-support jobs (30 and 6 percent, respec- 
tively), and a correspondingly higher proportion were employed in fabri- 
cation and assembling/handworking positions. Most of these experimental- 
control differences are related to the differential incidence of regular 
job-holding, as is illustrated by comparing Table IV. 5. A with Table IV, 5. B. 

It is interesting to note that both the experimentals and the 
controls had a fairly good record of job-holding. Among those who were 
employed in regular jobs in month 15, 75 to 80 percent were employed in 
month 22, and 64 to 70 percent were still with the same employer. The 
tenure rates for those in any job range from 54 to 77 percent. The 
slightly lower rates of job tenure among experimentals relative to controls 
are due primarily to the transition of some experimentals from STETS during 
this period. 

C. PROGRAM EFFECTS BY SELECTED SUBGROUPS OF THE TARGET POPULATION 

In an effort to better understand the mechanism through which STETS 
mitigates the employment problems of mentally retarded young adults and to 
help refine plans for targeting future interventions toward this popula- 
tion, we have examined estimates of program impacts for a large number of 
subgroups. In order to strengthen our understanding of the actual 
influence of a particular STETS project or of a participant's having a 
certain attribute on the impacts of the program, the net impact estimates 
presented for sample subgroups control statistically for differences among 
the subgroups in terms of factors other than those which define subgroup 
membership. For example, the net impact estimates reported for Cincinnati 
assume that the experimentals and controls in that site were similar to all 
sample members except for their city of residence; similarly, the results 
for males represent the difference in the expected value of the outcome 
measures for males in the experimental and control groups, assuming they 
have the sample mean values of all other characteristics. As was noted in 
Chapter III, because of both the limited sample size and the large number 
of subgroups of interest, the statistical powLr of this subgroup analysis 
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is low. For this reason, we have tended to focus our discussion on 
■ P^"^^"^ °f results, especially in the postprogram period (months 15 and 
22), and we caution the reader that these results should be viewed as 
tentative conclusions that warrant further substantiation based on future 
implementation and/or evaluation efforts. 

As indicated by these results (which are reported in Tables IV.6.A 
and IV. 6. B), estimated program impacts vary substantially across the five 
sites and among subgroups of the target population as defined by their 
personal and demographic characterisLics. Furthermore, the patterns of 
these differential impacts tend to vary between the largely in-program 
period (month 6) and the largely postprogram periods (months 15 and 22) of 
observation. 

!• Differences Across Sites 

In month 6, sizeable differentials existed in the estimated net 
impacts on regular and any job-holding across sites (see Tables IV. 6. A and 
IV.6.B)~differentials that can be traced largely to the variations in the 
nature and extent of program activities at that time. Of particular note 
is the relatively large (43 percentage point) impact on the likelihood of 
participants in Tucson holding any job in month 6~a result that is 
attributable in part to the higher proportion who were still in their STETS 
training jobs (see Table III. 3 and Riccio, 1984, Table 3.2, p. 59). in 
contrast, St. Paul exhibited the largest impacts (31 percentage points) in 
regular job-holding, where experimentals were transitioned out of the 
program sooner than average, and where virtually none of the control group 
members held such jobs. 

In months 15 and 22, when the results reflected primarily post- 
program impacts, St. Paul was the most effective site in moving target 
group members into competitive jobs (see Table IV.6.A).^ In month 15, 29 
percent of the experimentals in St. Paul were employed in such jobs, com- 



A ,?"^S^°"P workshop participation, reported in Appendix 

Table A. 13, show that the large positive increase in regular job-holding in 
bt. Paul was offset by a larger reduction in workshop participation. 
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•Statistically significant at the 10 percent level, two-tailed test. 
•^Statistically significant at the 5 percent level, two-tailed test. 
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pared with only 14 percent of the controls. Cincinnati is the only site in 
which the program seemed to have increased overall employment by as late as 
month 22 (by an estimated 15 percentage points); experimentals in this 
site also increased their incidence of regular job-holding by a similar 
amount. A notable observation in terms of this result is that overall 
employment among controls in Cincinnati was very low (13 percent), and 
almost none of this employment was in regular jobs. At the other extreme, 
the estimated post-program effects on regular job-holding in Tucson were 
relatively small (4 and -1 percentage points in months 15 and 22, 
respectively). 



2. Differences Among Other Sample Subgroups 

Judging from the results reported in Table IV. 6. A, it seems that 
STETS was most effective in increasing regular job-holding among four 
sample subgroups: those with mild or moderate retardation and whose 
retardation has organic causes, older individuals, males, and those who are 
more independent, as evidenced by their living arrangements and money- 
management ak.il Is— subgroups that tend to have average or lower-than- 
average probabilities of securing regular jobs on their own. Of these 
results, two sets are especially thought-provoking— the results on IQ 
levels and the results on gender. Estimated program impacts on the 
probability of holding a regular job are essentially zero for those 
participants with borderline IQ levels, 12 to 13 percentage points for 
those with mild retardation, and 23 to 28 percentage points for those whose 
IQ scores indicate moderate retardation. The net effect of these differ- 
ential program results is that STETS tended to raise the employment 
prospects for the mild and moderate retardation subgroups, from levels well 
below those for the borderline retarded group to roughly similar levels. 
For example, in the absence of STETS, only 11 percent of the moderately 



1 

The cross-site comparisons of results for earnings outcome 
measures are qualitatively similar (see Appendix Table A.4). 
2 

Large estimated impacts on any job-holding in month 15 are 
atuributable largely to STETS training jobs. 
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retarded participants would have been expected to hold a regular job in 
month 22, compared with 29 percent of the borderline group; subsequent to 
STETS, 38 percent of the moderately retarded sample held such jobs, com^ 
pared with 34 percent of the experimentals in the borderline group. 

One aspect of the estimated differential results for males and 
females is noteworthy — that STETS had large positive effects (14 and 17 
percentage points in months 15 and 22, respectively) on males and essent- 
ially no effects on females, despite roughly equal probabilities that males 
and females in the control group would hold regular jobs in the postprogram 
period (16 to 20 percent). There is no clear explanation for this lack of 
program impacts for females. However, it is notable that the STETS 
programs, themselves, were less successful in serving women than men. As 
was discussed in Chapter III, a much higher proportion of the males who 
entered the program advanced to Phase 2 jobs (73 percent of the males 
versus 57 percent of the females), and the males tended to spend 
substantially more time in these jobs (as was shown in Table III. 3). These 
findings are consistent with the reports by program operators that 
developing Phase 2 slots for females was somewhat more difficult than 
developing slots for males. However, this issue clearly warrants further 
investigation. 

Table IV. 6. B clearly shows that the overall null postprogram 
effects on the probability of holding any paid job also pertain to nearly 
all sample subgroups. The few hints that STETS may have increased overall 
levels of paid employment are associated with Hispanics, males, those with 
a higher-than-average degree of social independence, those whose 
retardation has organic causes, and those without significant recent 
workshop experience — all of which are groups that also exhibited higher- 
than-average increases in regular job-holding in month 22. 

3. Differential Effects by Key Program Features and Characteristics 

An important issue in assessing the STETS program model is the 
extent to which the nature or intensity of the program treatment affected 
the outcomes. This issue can also shed light on the male-female 
differences discussed above and may provide insight for monitoring and 
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assessment purposes to identify short-term indicators of program 
performance. As is evident from the ongoing efforts at the federal level 
to refine employment-related program models and to define perform^ace- 
monitoring criteria, this set of evaluation issues is difficult to 
address. Nonetheless, because of their importance, we have chosen to 
examine program effects that are associated both with three dimensions of 
program treatment (degree of program maturity at enrollment, program 
components entered, and hours of paid program-subsidized employment) and 
with one dimension of short-term performance (entered employment at 
termination). 

For practical reasons, the demonstration design did not incorporate 
the random assignment of participants to various configurations of program 
services. Thus, the analysis becomes more complex and the results more 
tenuous, since elements of nonrandom self-selection and program assignment 
of individuals to various program treatments will undoubtedly have been 
present. In cases where nonrandom selection or assignment of individuals 
to program treatment occurs, it is likely that controlling for measured 
differences in personal characteristics will not fully account for the 
differentials in the expected performance of participants who receive 
various program services in the absence of the program treatment. Thus, if 
a typical impact regression model such as has been used to estimate the 
overall and subgroup analysis were used, the coefficients on the program 
treatment variables would measure the true effect of the particular program 
treatment, as well as the effects of unmeasured characteristics that affect 
both the self-selection or assignment to various treatments and the 
outcomes of interest. 

The procedure we adopted for dealing with this problem in analy- 
tical terms was to use^ the instrumental variable procedure proposed by 
Maddala and Lee (1976). In applying this procedure, we first estimated 
models of the probabilities that STETS participants entered the various 



1 

Alternative procedures, such as those proposed by Heckman (1979) 
and Heckman and Robb (1983), could have been adopted with roughly 
equivalent results, 
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program components, and that they entered employment upon leaving the 
program, and estimated a model to predict the length of time they spent in 
paid subsidized employment. We then used predicted measures of theue 
program variables, rather than actual values, in the analysis. Because 
program maturity at enrollment was exogenous, estimating the impacts of the 
program on individuals who were enrolled at various times relative to the 
stage of program maturity did not necessitate using special estimation 
techniques. 

Table IV. 7 summarizes the estimated overall program impacts for 
subgroups of experimentals defined by these three components of their STETS 
participation and the one component of the program's immediate outcome. 
These results suggest that an ongoing program would be expected to have 
greater impacts than did the STETS demonstration, which seemed to have had 
no beneficial impacts on those who enrolled during the start-up or phase- 
down periods of the programs. It Is especially noteworthy that the 
programs tended to be quite effective for males, regardless of when they 
entered the program (see Table IV. 8). Furthermore, the programs achieved 
substantial success with the females who enrolled during the "steady-state" 
period of program operations; the problem in achieving significant net 
impacts for females was concentrated in the periods when the programs were 
in transition (i.e., gearing up or phasing down), and it is these 
difficulties that account for the overall findings of no impacts during the 
early and later periods of program operations. 

The results also indicate that relatively long periods of sub- 
sidized training jobs are important to the success of the STETS concept. 
Finally, the analysis provides some suggestion that "entered employment 
rates" can be used to gauge the success of programs. However, it is 
noteworthy that the sites which exhibited the largest impacts on employment 
in regular jobs were not always the same sites which exhibited the highest 
incidence of entering employment at the time individuals left the STETS 
program. For example, at one extreme, St. Paul exhibited the largest 
employment impacts and the lowest enter ed-employment rate; and, at the 
other extreme, Tucson exhibited no impacts on employment by month 22, but 
achieved the highest entered-employment rate. Thus, it seems obvious that 
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TABLE IV.7 



ESHHMED PROGRAM imClS ON ETFLOV^ENT ACHVITY 
IN MONTH 22, BY PROGRAM EJPERIENCES 



Outcome Measure 



Subgroups Defined by 
Prograro Experleooes 



Percent of Average Weekly Average Wedcly 

Experlmentals Percent In Percent in Earnings Earnings 
In Subgroups Regular Job^ Paid Job d-^.i^^ i^i^a 



Regular Dobs'' 



All Dobs 



Stage of Rro^an 
C^pmtlDie At Ehrollmit^ 

Eavly 21.6 

Steady state 60.3 

Late 18.1 

fVo^an Components Entered 

Phase I only 33.6 

Phases I and II 66.'f 

HtMCs of 1^ STETS 

< 500 hours 30.9 

V 500 hours 69.1 

TexBlnatlon 

Entered unsubsldlzed 'f3.8 

enployment 

Other 56.2 



OA 

11.1 



3.9 
16.3*^* 



■0.5 
17.3* 



aB.5»» 
-0.1 



-m.1 
5.0 
3.2 



-2.9 
13.8» 



-22.^^ 
17.3 



20.6»» 
-16.0* 



9.36 
20.92** 
^^.19 



2.58 
21.76** 



18.50 
22.28** 



at;.92** 

0.71 



6.06 
15.17** 
2.09 



-9.8 
21.05** 



-17.80 
22.21* 



37.98** 
-9.57 



Nunt)er in Sanple 



^3 



^3 



402 



m 



NOTE: All of these estJjnates, with the exception of those pertaining to stage of progran operatiore at the 

time of enrollment, were estimated using Instrumental variables procedures such as have been proposed by 
Maddala and Lee (1976). The integrity of these results necessarily depends on the validity of 
untestable assunptlons that underlie this approach for correcting for selection bias. For this reason, 
we offer these results more as exploratory findings that might provoke firther examination within the 
context of future research and/or program efforts. 

^egi iar Jobs are those that ars neither tralntng/work-stLidy nor workshop/act Ivlty-oenter Jobs. 

^ogram operations were divided Into three periods: the early period In which the progran was beginning 
operations, the "steady state" period In which program size and operations seoned rea5ondi)ly st^le, and the 
late period In nhich program enrollment was declining. The enrollment period corresponding to the steady 
statf Is a five-fflonth period defined by site as follows: Mew York and Tucson (January to M^ 1982), 
CinclfMtl (March to aily 1982), St. Pajl (April to August 1982), and Los Angeles (May-Septenter 1982). 

^Statistically significant at the 10 percent level, two-tailed test. 
^♦Statistically significant at the 5 percent level, two-tailed test. 
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TABLE IV. 8 



ESTIMATED PROGRAM IMPACTS ON THE PERCENT HOLDING A REGULAR 306 AND ANY 
aOB IN MONTH 22, BY STAGE OF PROGRAM OPERATIONS AND GENDER 



Stage of Program 


Percent Holding a Regul 


ar aob 


Percent Holding Any Paid 3ob 


Operations at Enrollment^ 


Males 


Females 


Total 


Males 


Females 


Total 


Early 


13.6 


-15.8 


OA 


21.1 


-16.7 


-H.I 


Steady State 


18.9»» 


n.i» 




32.3*» 


1.3 


5.0 


Late 


21.6 


-10. ^ 


11.1 


20.1 


-36.6* 


3.2 


Number In Sample 


228 


175 


¥)3 


228 


175 


^3 



^eguler Jobs are those that are neither tralnlng/work-study nor workshop/act Ivlty-center 
Jobs. 



Program operations were divided Into three periods: the early period In which the progran 
was beginning operations, the "steady state" period In which program size and operations 
seemed reasonably stable, and the late period In which program enrollment was declining. The 
enrollment period corresponding to the steady state Is a five-month period defined by site as 
followst New York and Tucson January to May 1982), Cincinnati (March to Ouly 1982), St. 
Paul (April to August 1982), and Los Angeles (May-September 1982). 

♦Statistically significant at :he 10 percent level, two-tailed test. 
♦♦Statistically significant av the 5 percent level, two-tailed test. 



89 



146 



entered-employment rates alone (i.e., without some adjustment for the 
characteristics of the client population served) should not be used as a 
key program performance measure. 

Taken at face value, the ^esults along each of these dimensions are 
quite strong. However, these findings should be considered only suggestive 
of the possible indicators of differential program effectiveness. Because 
of the inherent weaknesses in the analysis, noted above ^ they might be most 
valuable in terms of guiding future evaluation efforts. 

D. SUMMARY AND CONCLUSIONS 

This evaluation clearly indicates that a STETS-type program can be 
expected to improve the employment prospects of mentally retarded young 
adults. A STETS-type program will induce this effect primarily by helping 
participants, transition out of workshop/activity centers into regular, 
competitive jobs. The results from this demonstration suggest that such a 
program can reduce the incidence of workshop jobs and increase the 
incidence of regular jobs by more than 60 percent (11 to 12 percentage 
points). These effects can be expected to be even larger in programs 
targeted toward subsets of the STETS target population, such as the mildly 
and moderately retarded and males. However, developing defensible target- 
ting strategies that Increase the proportions of the client population from 
these groups would be difficult, if not impossible. A more promising 
approach for improving program effectiveness would seem to be to provide a 
better understanding of and to mitigate the problems in serving females, 
and, relatedly, to increase the incidence of Phase 2 employment among all 
participants, but especially among females. Given the similarity of 
results for males and females during the "steady-state" period of program 
operations, it seems likely that this goal can be attained. 
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V. IMPACTS ON TRAINING AND SCHOOLING 



ERIC 



An examination of the impacts of STETS on the likelihood that 
mentally ret*.rHed young adults will enroll in training or school over time 
is important for two reasons. First, STETS was largely an employment- 
training program, whose purpose was to move individuals out of training 
programs and schools into competitive employment. Thus, a complete 
assessment of the program should measure its success in reducing the long- 
term use of training and education programs. Second, since providing these 
types of services tends to be quite expensive, program-induced changes in 
participation in them can significantly affect the overall benefit-cost 
assessment of the program. 

At the outset, we should note that the discussion in Chapter IV on 
the effects of the program on training and workshop job-holding and the 
discussion in this chapter on the effects of the program specifically on 
training overlap to a considerable extent. Overall, 83 to 93 percent of 
all training for control group members included some type of job 
component. In Chapter IV, we focused on regular job-holding, with training 
and workshop jobs constituting the residual categories of job-holding. In 
this chapter, we focus more directly on the impacts of the program on 
training itself. 

We begin this chapter by providing an overview of the expected 
nature and level of training and schooling for STETS participants during 
the observation period had they not enrolled in the program. In Section B, 
we discuss the overall impacts of the program on the likelihood of 
enrolling in training or schooling. In Section C, we describe key 
differences in the impacts across sites and among sample subgroups defined 
by demographic and personal characteristics. The final section summarizes 
the key findings from this portion of the analysis and notes their main 
policy significance. 

A. TRAINING AND SCHOOLING IN THE ABSENCE OF STETS 

The behavior of the control group at baseline and at each of the 
follow-up waves (see Table V.l) indicates that 30 to 39 percent would have 
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TABLE V.1 

SCHOOLING AND TRAINING OF CONTROL GROUP MEMBERS, BY SITE 



Site 



Activity iind 




Los 


New 


St. 






Time Period 


Cincinnati 


Angeles 


York 


Paul 


Tucson 


Total 


Percent in Any Training 














Baseline^ 


16.7 


40.5 


44.9 


14.8 


27.9 


30.5 


Month 6 


20.7 


44.4 


44.4 


50.0 


38.7 


39.0 


Month 15 


25.0 


28.2 


29.2 


46.2 


27.3 


29.9 


Month 22 


23.7 


26.3 


21.7 


42.3 


38.6 


29.7 


Percent in Training 














with a Dob Coaponent 














Baseline^ 


1^^.3 


35.7 


34.7 


14.8 


23.3 


25.6 


Month 6 


20.7 


40.7 


41.7 


50.0 


32.3 


36.2 


Month 15 


20.5 


23.1 


29.2 


42.3 


27.3 


27.4 


Month 22 


21.1 


21.1 


19.6 


42.3 


25.6 


24.6 


Percent in Any School 














Baseline^ 


46.5 


16.7 


37.3 


11.1 


51.1 


34.8 


Month 6 


17.2 


29.6 


21.6 


11.1 


12.9 


19.0 


Month 15 


9.1 


7.7 


14.6 


0.0 


18.2 


10.9 


Month 22 


7.7 


18.4 


8.7 


3.7 


20.5 


12.4 


Percent in School 














with 3ob Component 














Baseline^ 


23.3 


2.4 


25.5 


0.0 


27.7 


17.6 


Month 6 


6.9 


0.0 


5.4 


0.0 


12.9 


5.6 


Month 15 


4.5 


0.0 


0.0 


0.0 


13.6 


4.0 


Month 22 


0.0 


5.3 


0.0 


3.7 


11.4 


4.1 


Number In Sample: 














Baseline 


43 


42 


51 


11 


47 


210 


Month 6 


29 


27 


37 


18 


31 


142 


Month 15 


44 


39 


48 


27 


44 


202 


Month 22 


39 


38 


46 


27 


44 


194 



NOTEj These data are unadjusted subgroup means. 

*In some cases, baseline Interviews were administered several weeks after random assignment. 
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been in training at any time, and that between 25 and 36 percent would have 
been in training that offered a job component. It is especially noteworthy 
that the incidence of training rose temporarily among controls, from 31 
percent at the baseline period to 39 percent at month 6. This rise can 
probably be attributed to the fact that the agencies in St. Paul and Tucson 
referred many of the control group members to non-STETS programs, most of 
which included job components. 

As shown in Table V.2, the characteristics of those who in the 
absence of STETS would have been more likely to enter a training program in 
the postprogram period parallel quite closely the characteristics of those 
who also would have been more likely to hold a paid job in this time period 
(see Table IV. 3 in Chapter IV). They are mildly or moderately retarded, 
their retardation has organic causes, they are 22 years of age or older, 
they are white, they live in supervised settings, and they have not 
recently held a regular job for three or more months. 

At the time of their referral to STETS, 35 percent of the control 
group were enrolled in schooling programs, half of which included a job 
component. As stiown in Table V.l, the school enrollment rates dropped very 
quickly subsequent to baseline, so that by months 15 and 22 between 11 and 
12 percent were enrolled in some type of school, and only 4 percent were 
enrolled in educational programs that offered a job component. By month 
22, nearly two-thirds of those who were attending school were in either a 
regular or a vocational secondary school, and 78 percent of those schools 
offered special curricula for mentally retarded students (see Table V.3). 

Notable differences across sites in terms of the incidence of 
school enrollment at referral parallel the differences across sites in 
terms of the proportion of referrals from the public school system (see 
Riccio and Price, 1984, Table 2.2, p. 33). However, only the school 
linkages in Los Argeles and Tucson show evidence of possible long-term 
influences on control group behavior; 18 percent of the controls in Los 
Angeles and 21 percent of those in Tucson were in school at the time of 
their 22-Tnonth interview (see Table V.2). 
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TABLE V.2 



PERCENT OF CONTROL GROUP MEMBERS IN TRAINING AND SCHOOL 
IN MONTH 22, BY CHARACTERISTICS AT BASELINE 



Subgroups Defined by 

Characteristics at Baseline Individuals In Training Individuals In School 

Total Saiq>le 29.7 12.5 
Site 

Cincinnati 23.7 7.9 

Los Angeles 26.3 18.4 

New York 21.7 8.7 

St. Paul 42.3 3.8 

Tucson 38.6 20.5 

IQ Level 

Borderline 18.3 11.7 

Mild 33.6 11.5 

Moderate 42.1 21.1 

Age 

Younger than 22 23.6 16.4 

22 or older 46.2 1.9 

Gender 

Male 27.1 11.2 

Female 32.9 14.1 

Bace/Btfanlcity 

Black 18.8 7.8 

Hispanic 20.7 6.9 

White atki other 39.4 17.2 

LiTlng Arraqgeaent 

Livlt^ with parents 28.4 14.8 

Living in supervised setting 50.0 0.0 

Living independently 18.2 0.0 

Financial Management Skills 

Indepet^ent 14.9 8.5 

Not independent 34.7 13.9 

lacelpt of Ttansfen 

SSI/SSDI 40.3 17.9 

Other transfers only 23.5 8.8 

No transfers 23.6 10.9 



Secoadarf Handlcapa 
Secondary handicap 
No secondary handicap 



34.4 13.1 

27.5 12.2 

Caaaa of latardatlon 

Organic 56.7 16.7 

Non~organic 24.7 11.7 

Benefactor 

Benefactor 41.4 13.8 

No benefactor 24.6 11.9 

Vork Bzperlenee In 
Teata Prior to Snrollaent 
Regular Job lasting 22.2 3.7 

>3 aonths 

Otlfer Job lastii^ 31.0 10.3 

>3 aonths 

Otnier 30.8 15.9 

School Statua at Referral 

Enrolled 31.5 24.7 

Not enrolled 28.6 5.0 



Number in Saaple 192 192 

NOTE: These data are unadjusted subgroup means. 
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TABLE V.3 



CHARACTERISTICS OF SCHOOLS ATTENDED BY CONTROL GROUP MEMBPf<S 
IN MONTH 22, BY SITE 
(Percent Distribution) 



Site 



School 

Characteristics 


Cincinnati 


Los 
Angeles 


New 
York 


St. 

Paul 


Tucson 


Total 


Type of School 














Secondary school 


66.7 




0.0 


0.0 


62.5 


37.8 


Vocational school 














Secondary 
Posts econ da rv 

1 WW V w www 1 1 VJU & T 


0.0 


0.0 

m-. 3 


100.0 
0.0 


100.0 
0.0 


12.5 
0.0 


26.1 


College^ 


0.0 


U.3 


0.0 


0.0 


25.0 


13.0 


Adult noncredlt 


33.3 


57.1 


0.0 


0.0 


0.0 


21.7 


Type or CurriculiM 














School offers special 
curriculum for the 


66.7 


7} A 


100.0 


100.0 


75.0 


78.3 


mentally retarded 














No special curriculum offered 


33.3 


28.6 


0.0 


0.0 


25.0 


21.7 


Number In School 


3 


7 




1 


9 


2^^ 



NOTE: Data on type of school were obtained from Interviews with agencies that offered schooling 
(and other services). Agency representatives were asked to specify the primary services 
they provided, which may differ In some cases from the specific .^rvlces that STETS sample 
members were receiving. The data are unadjusted subgroup means. 



'These include community and Junior colleges. 
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ihose who were more likely to be in school during the postprogram 
period differ in two important dimensions from those who were more likely 
to be in training: they are younger than age 22, and they live with their 
parents. The age result simply highlights the fact that the vast majority 
of schooling-status changes involved leaving rather than enrolling in 
school. It also undoubtedly reflects the influence of P.L. 94-142, which 
mandates that schools provide free and appropriate education to mentally 
retarded individuals until they reach the age of 22 (see Chapter I). 

B. OVERALL PROGRAM EFFECTS 

In this section, we discuss the effects of the program on the 
incidence of training and schooling both during the in-program period and 
subsequent to program termination. As noted in Chapter IV, we have 
approximated these time periods by using the 6-raonth follow-up results as 
the basis for measuring the in-program effects, and the 15- and 22-month 
interview data as the basis for estimating the postprogram effects. 

1. In-Program Effects 

The influence of STETS training jobs is evident in the large 
positive increase in the incidence of training among experimentals relative 
to controls in month 6 (62 versus 41 percent), 86 percent of which arose 
from increases in training that offered a job component (see Table V.4). 
The impacts on school enrollment are negative, and all are substantially 
smaller than the training effects in absolute size. In month 6, only 8 
percent of the experimentals, compared with 16 percent of the controls, 
were in school, and only 1 percent of the experimentals were in a school 
program that offered a job component. This finding undoubtedly reflects 
the fact that STETS directed the activities of individuals toward job 
training and away from other activity statuses (including school). 

2. Postprogram Effects 

The 15- and 22-month results reported in Table V.4 show that STETS 
reduced the incidence of both training and schooling during the postprogram 



96 

153 



ESnmTH) PfttGRAH IM>ACT5 ON TRADaNC AK> STHOCUNS 



North 6 Month 15 Month 22 

Control Cbntrol Cbntrol 

Experl»r««l Qroi^ Estl«tt«d Experli»nt«l Qrai^) EstlMtod Experijwntil Qpoi^ EstUit«l 

OitooweHMsuws ttw Hwn Mean Iipact Qrcut. Hmh Mpan Tmpatit Group Me« hey Iwpact 

RMOtnt In any trtlnlng 61»7 40.6 21J»» 20.6 28,4 -7.8» %.6 29J -12.5« 

Rarowt In training 55.1 J7.0 18.1»» 18.2 26.7 -8.5»* 13.5 24.9 -11.M» 
with a >>b o iM p tfwi l 

Ptrotnt for ihoa 6.6 2.5 4.1* 2. i 2.6 «0.5 2*9 2*9 0.0 
training waa tho 
■iln activity 

SohooUJiQ 

fMrcBft In any school 7.5 15.7 -8.2»» 6.2 10.1 -3.9 8.0 11.4 -3.4 

Pteroent In school 1.1 5.4 -4.3»» 2.2 3.7 -1.5 2.0 4.6 •2.6» 
with job oa yo nai l 

PercBft for irfm 1.2 8.4 -5.2» 3.2 5.9 ^.7 6.3 6.4 -0.1 
school was the 
■iln activity 



Njiter in S«pls 283 402 395 

nsrosnt of ExpsrlMontals 

in Phaw X and Rmm It 67.8 18.3 0.0 



NOTEt Those r«sults were estinrited t^rau^ ordinary least square tecNiiqm. Definitions an! mw of control varlAles that ve included in 
the Models are presented in Appendix TAle A.1. FuU results fnm a representative set of the Ij^pact equation that mderUe these 
rsMilts aro preMnted in Appendix Tables A.7 Ml A.8. 

*^istically sigrdficsnt at the 10 peroent level, tw>-uiled teet. 
^^Statistically significant at tho 5 peroent level, tno-tailed test. 
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period. However, only the effects on training are consistently large and 
statistically significant. 

In month 15, when 18 percent of the experimentals were still in 
STETS, 21 percent of the experimentals, compared with 28 percent of 
controls, were in training programs — a 7 percentage point differential. By 
month 22, the program-induced differential in the percentage who were in 
training had risen to 13 percentage points. In both time periods, these 
reductions were associated almost entirely with training that included a 
job component — primarily training in sheltered workshops and activity 
centers. Thus, the reductions have no overall effect on the likelihood 
that training rather than employment was the main activity of sample 
members. 

C. PROGRAM EFFECTS BY SELECTED SUBGROUPS 

In terms of the training and schooling results, the primary reason 
to examine program effects across sample subgroups is to develop a better 
understanding of the nature of the secondary consequences of the STETS 
intervention on the use of education and training resources. The a priori 
expectation was that the effects on such use would derive directly from the 
influence of the STETS treatment and from the program-induced changes in 
the incidence of competitive employment. 

These expectations on the differential program effects were not 
fulfilled, as evidenced by comparing the results presented in Tables V.5.A 
and V.5.B with those that were presented in Tables IV. 6. A and IV. 6. B in 
Chapter IV. With a few exceptions, no noteworthy patterns of subgroup 
effects emerge for the training and schooling outcomes. Among the 
exceptions is the fact that the in-program effects on training are 
concentrated in three sites: Cincinnati (25 percentage points). New York 
(30 percentage points), and Tucson (43 percentage points). They also tend 
to be larger among subgroups which had lower probabilities of enrolling in 
training in the absence of the STETS demonstration: those who are younger 
than age 22, Hispanics, those who do not live in supervised settings, and 
those who are enrolled in school at the time of their referral. 
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lfa£ V.5 



£SnmTH) PRQCRAH DPAaS ON TRAIHirC AK) SCHOCLING 
FOR KEY SJBOnOUPS OF STETS P^CIPWTS 

A. PERCENT IN TRAINtNC 







Month 6 






)t>nth 15 




Month 22 








Ovitrol 






Cbntrol 






Control 




by Char«cterftJtlcs 


experiment «1 


Qrou|> 


EstlMted 


Experimental 


Qroup 


EstUated 




icoup 


Eetlfaatfld 




(Voup HbMX 




Lipact 


Qroup Mwn 






Qroup Mean 


Me«) 


Inpact 




61.7 


¥).6 


21.1»» 


20.6 


2B.4 


«7.8» 


16.6 


49.1 


-12.5»» 


Site 




















(.incinnKi 






Ok. c 






25.8 


11 ^ 




-14.5 


U» Angelas 




41.0 


J^.9 




27.6 


-1.6 


IS X 


9A S 


-12.9 


Nm York 


7S.3 


M.O 




13.1 


29.0 


.15.9* 


lb A 


?A a. 


-13.6* 


St. p«a 




45*2 


-2.7 


9.4 


48.8 


.39.4»» 


24.2 


34.9 


-10.7 


Tucson 


8S.0 


43.0 


43.0^ 


32«4 


20.6 


11.8 


22.1 


30.0 


-7.9 


Borderline 


53.2 


3S.9 


14.3 


12.9 


22.8 


-9.9 




77 n 


-7.4 


NUd 




40*2 


24.2<^ 


22.8 


28.7 


-5.9 


17.4 




-16.»» 


Hoderete 




^.7 


22.6 


2B.5 


41.7 


-13.2 


21.9 


24.3 


-2.4 






















Younger than 22 


fi.2 


3^9 


Z7.3»* 


19.4 


22.6 


-3.2 




22.5 


-7.3 


22 or older 




56.2 


4.2 


23.1 


41.3 


.18.2<^» 


19.7 


43.6 


-23.9W 


Hile 




40.2 


16.7»» 


21.2 


30.3 


-9.1 






-10.5* 


F«Mle 




41.1 


26.9« 


19.6 


25.7 


-6.1 


14. J 




-15.0»» 


























%.5 


21.9* 


18.6 


Z7.1 


-8.5 


12.4 


23.3 


-10.9 


HUptfOc 




31.9 


37.9*»* 


20.8 


28.2 


-7.4 


12.0 


16.6 


^.6 


Ihite «d other 


fi.O 


4I»5 


17.5»» 


21.4 


29.1 


-7.7 


20.2 


35.6 


-15.4** 


LhSng'irtSrpeMnte 


^.2 


40.1 


21-,'!»» 


18.9 


30.6 


-11.7»» 


16.9 


309 


-14.0^ 


Living in mipervieed 


79.0 


65.6 


13.4 


50.2 


39.1 


11.1 


26.0 


31.5 


-5.5 


setting 










a 










Living independently 


45,1 


H.0 


31.1 


8.9 


0.0 


8.9 


2.5 


6.5 


-6.0 


rtanoM NmvMi* SkUU 




















ZndsporJoffc 


€7.6 


44.7 


22.9*»» 


20.8 


Z7.3 


-6.5 


11.6 


22.4 


-10.6 


noc inoepenaenc 


59.3 


36.9 


ao.4»» 


20.3 


28.6 


-8.5» 


18.5 


31.6 


-13.1»» 






















sa/ssDi 


<5.7 


51.1 


14.6 


22.8 


3fi.2 


-13.4* 


23.0 


31.6 


^.6 


Other trenefiere only 


70.3 


41.8 


28.5»» 


18.3 


27.6 


-9.5 


11.1 


31.0 


-19.9^ 


No tenfut 


51.9 


30.9 


21.0« 


20.3 


21.3 


-1.0 


15.6 


24.7 


-9.1 



tfHit^daitto 

Organio 
Non-organlo 

Mcfi^pKUNW In Im 
VenMarteEmUKi* 

Regular job lasting 

>3Honths 
OAr job Ustim 

>3k 



46.7 
64.6 



34.5 
41.8 



12.2 
22.d«»» 



27.8 
19.0 



29.2 
28.3 



«0.4 
-9.3« 



22.6 
15.3 



48.4 

25.1 



-25.6»* 
-9.8»» 



61.4 


40.0 


21.4 


18.1 


18.9 


«0.8 


5.9 


27.7 


-21.6* 


A.4 


48.7 


15.7 


24.7 


3S.8 


-11.1 


19.3 


as. 5 


-7.2 


9.8 


3^6 


25.0»» 


18.3 


26.0 


-7.7 


17.9 


31.2 


-13.3»« 



Enrolled 0.5 y^Z 29.3»» 19.9 34.0 -14.1* 13.7 29.2 -15.5* 

Not enrolled 60.9 43.4 17.5»* 20.6 25.6 -5.0 18.0 29.1 -11.1** 



Hflter in Sa|>l0 263 402 395 



NQITEt These results Mre estiMted throuj^ ordinary leait squares techniques. In addition to the control varlAles that are included in the 

aodels which uvterlie the overall net ii^act estisMtas reported in TAle V.4, these nodels ineluds variables thet interacted the treateent 
varlAle with the subgroup varlAlee. The fUll regression reailts that underlie the 6- and 22-«>nth results ar« presented in Appendix 
Tdblst A.7 and A.8. 

a 

The control Ttiup "Ban value was actually calculated to be sli^^tly negati\« because o^ the liprecision of QLS estliaatlon with a binary outoaae 
varldl>le. 

•Statistically significant at the 10 percent level, tm>>tailed test. 
'^•Statistically sigrtificMt at the 5 perowit level, two-tailMi test. 
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TtfUE V.5 



EsnmTED Pfmm wacts on traininc aw soolinc 

FOR SUBQfCUPS OF STETS P^nOPWTS 
B. PCRGEKT IN SCHXL 







Nonth6 






Nonth 15 




r^nth 22 




Sub||roup» OaflMd 




CorArol 






Oontrol 






Ckxtrol 




by Char«ct«rlatlo 


Experlflental 


Qrai4> 


EstlMted 


Experimental 




EstlAited 


Experiaental 


Qraip 


EstlMted 




Qraup Hean 


NMn 


Impact 


(ktM) Hean 


Hean 


lapact 


Qraup Hean 


Hean 


Lepact 




7.5 


15.7 


-8.2»» 


6.2 


10.1 


-3.9 


8.0 


11.4 


-3.4 


Stt* 

ClJidnnatl 

Nbw York 
St. Pail 
Tucion 


3.6 
9A 
8.5 
7.8 
8.1 


9.7 
27.4 
12.8 
17.6 
12.6 


-18.0* 
•4.3 
-9.8 
^.5 


5.1 

lU.O 

4.1 
4.9 

6.2 


9.5 
111 
15.5 
-3.0 
11.3 


•4.4 
^.5 
-11.4* 
7.9 
-5.1 


XO 
15.8 
3.6 
3.3 
9.5 


4.6 
22.8 
9.5 
2.8 
13.4 


-1.6 
-7.0 
-5.9 
0.5 
-3.9 


Borterllne 

HUd 

Hodente 


12.7 
4.6 
9.2 


11.4 
18.7 
11.9 


1.3 
-14.1»» 
-2.7 


6.3 
6.1 
6.9 


15.1 
8.1 
7.5 


-8.8» 

-2.0 

-0.6 


5.2 
10.5 
1.6 


9.6 
10.2 
22.3 


-4.4 

0.3 

-ao.7»» 


22 or older 


7.5 
7.4 


15.3 
16.8 


-7.8 
-9.4 


6.8 
4.8 


11.0 
8.0 


•4.2 
-3.2 


8.9 
5.9 


15.5 
2.2 


-6.6* 
3.7 


Hale 
FMle 


8.4 

6.4 


15.3 
16.3 


-9.9 


4.2 
9.0 


7.9 
13.1 


-3.7 
-4.1 


1.8 
10.9 


t).9 
12.0 


-9.1 
-1.1 


Wilts «nd other 


7.7 
2D.5 
4.9 


16v0 
16.2 
15.4 


-8.3 
4.3 
-10.5* 


4.1b 
-0.0 
4.9 


6.9 
14.8 
14.3 


-2.8 
-14.4^* 
-9.4« 


10.5 
4.1 
7.6 


5.5 
OJ^ 
17.5 


5.0 
3.7 
-9.9»» 


Living with pirentt 
Living In ajpervlaed 

setting 
Living Independently 


7.2 
5.0 

22.9 


19.5a 
0.0 

3.0 


-12.3»» 
5.6 

19.9 


7.7a 
0.0 

a 

0.0 


11.5 
7.5 

0.7 


-3.8 
-7.5 

-0.5 


9.2 
-6.0 

10,7 


13.8 
-1.6 

3.5 


-4.6* 
-4.4 

7,2 


FlmwdAt WwiB— » ailto 

Independent 
Not Independent 


5.2 
8.5 


2S.5 
11.3 


-21.3»» 
-2.»« 


9.6 
5.0 


11.5 
9.6 


-1.9 
-4.6 


9.4 
7.5 


10.8 
11.6 


-1.4 
^.1 


Keoelpt or Trenefto 

S9/SSDI 

Other transfers only 
No tranefsre 


1.0 
11.4 

10.0 


15.1 
12.8 
19.4 


-14.1 
-1.4 
-8.4 


4.3 
5.5 
8.8 


2D.5 
1.9 
7.6 


-16.2 
3.6 
1.2 


11.3 
7.3 
5.3 


17.1 
8.6 
8.3 


-5.8 
-1.3 
-3.0 


Cm* tflliUrdetlMt 

Organic 
Non-organle 


8.5 
7.3 


23.0 
14.3 


-14.5 
-7.0 


7.2 
6.1 


3.5 
11.4 


3.7 
-5.3» 


4.5 
7.7 


7.0 
12.3 


-2.5 
-4.6 


Vxtc &paflanoe in Teo 

Tm VMra Mer to Emltertt 

Regular >>b lasting 

>3 aonths 
OtiJvr >>b lasting 

>3 Months 
OAr 


6.9 
3.6 
10.6 


11.6 
18.0 
15.3 


-4.7 

-''i4.4»» 

A.7 


1.3 
6.1 
7.8 


7.6 
10.6 
10.5 


^.5 
-2.7 


2.1 
10.0 
8.3 


3.67 
10.7 
14.0 


-1.6 
-0.7 
-5.7 


SriMl StaiM A Ranaml 

Enrol bd 
Not enrolled 


7.2 
7.6 


24.1 
12.0 


-16.9*« 
-4.4 


5.6 
6.5 


17.6 
6.7 


-12.0* 
-0.2 


7.4 

6.3 


18.9 
8.0 


-11.5* 
0.3 


NLMter In Smpl» 






283 






402 






395 



WTEt These remits Mre estlavted through ordinary leait squares techniques. In addition to the oontrol varlAles that are Included in the 

Models uhlch underlie the overall net ^pact estlMtes reported in Table V.4, these aodeXs Include variables that Interacted the treatnent 
variable with the aUbgraup variM>les. The fUll regression results that underlie the 6- and 22HK>nth results are presented in Appendix 
TdE>les A.7 «vl A,8. 

a 

The control group «ean value was actually calculated to be slightly negative because oT the li^reclslon oT OLS estlMtion with a binary outcoae 
variable, 
b 

The Mperimntal group Mean value was actually calculated to be slightly negative because of the ^rocision of CLS estlMtion with a binary 
outcoae variable. 



•Statistically signified at the 10 percent level, two-tailed test. 
*«StatUtioally slgnificsft at the 5 percent level, tMHUiled test. 
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During the postprograra period, a few noteworthy patterns of 
subgroup results emerged^ First, consistently larger reductions in 
training were observed among individuals who were 22 years of age or older, 
among individuals who were living with their parents, and among individuals 
who were enrolled in school at the time they were referred to STETS — 
subgroups that also tended to have experienced program-induced increases in 
their incidence of regular job-holding. Second, the program-induced 
reductions in school enrollment tended to be more consistently evidenced 
among whites and individuals who were enrolled in school at the time of 
their referral. 

D. SUMMARY AND CONCLUSIONS 

The combination of the STETS treatment itself and its effectiveness 
in increasing the incidence of competitive employment in the postprogram 
period had the expected effects of increasing the use of training programs 
(which offered job components) and of reducing the use of schooling during 
the in-program period and the use of both training and schooling during the 
postprogram period. Obviously associated with the reduction in the use of 
training and schooling programs is a social benefit in terms of lower 
resource expenditures (see the discussion on the benefit-cost analysis and 
its results in Chapter VIII). However, what we cannot answer from this 
demonstration is whether the STETS experience of those individuals who 
would have used more regular training and/or schooling had they not 
enrolled in STETS at least compensated for the foregone human capital from 
the non-STETS programs. 
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VI. IMPACTS ON PUBLIC TRANSFER DEPENDENCE 



As illu8trate«.I by the impacts of STETS on the employment and 
training outcomes that were described in Chapter IV, the program did 
achieve some notable success in integrating mentally retarded young adults 
into the competitive labor market. These employment outcomes would be 
expected to have secondary effects on public transfer dependence, since the 
receipt and amount of most types of transfers are income-conditioned. In 
this chapter, we present the impacts of STETS on public transfers — both 
cash and in-kind transfers. Cash transfers received by sample members 
primarily include Supplemental Security Income (SSI), Social Security 
Disability Insurance (SSDI), Aid to Families with Dependent Children 
(AFDC), and general assistance. In-kind transfers include food stamps. 
Medicaid^ (in Tucson, the state equivalent). Medicare, and subsidized 
housing. 

We should first clarify several points about the process whereby 
measured and defined transfers. First, survey respondents seemed to be 
able to report receiving SSI or SSDI, but were often unable to distinguish 
between the two. Because of the similarities between the two programs, we 
have simply combined the two. SSI recipients generally account for over 80 
percent of the combined category of recipients. Second, we encountered the 
same problem for Medicaid and Medicare, which are usually (but not always) 
associated with, respectively, SSI and SSDI. Again, we have combined 
recipients of these two transfer sources into one category. Third, we 
encountered limited use of a variety of living arrangements. We construct- 
ed the category of subsidized housing to include living in group homes, 
supervised apartments, and institutions. (This concept is also considered 
in Chapter VII as a measure of Independence.) Finally, several types of 
transfers — primarily welfare and fod stamps — are likely to be associated 



1 

These various income programs are described ind their eligibility 
criteria are documented in The Social Security Bulletin, Annual Statistical 
Supplement. Washington, D.C.: U.S. Department of Health and Human 
Services, 1983. 




with a combined family or household unit, rather than vd.th the sample 
member only. We credited the sample member with having received such types 
of transfers only if the sample member (or a proxy respondent) reported 
that he or she was part of the administrative unit for each respective 
type. 

We begin this chapter by providing an overview of the expected 
transfer use of STETS participants had they not enrolled in the program. 
This overview shows the receipt of each type of transfer both for the 
entire sample and for several important subsamples, and shows the amount of 
each type of cash transfer. Section B presents the estimated overall 
program impacts on transfer dependence, and Section C discusses differences 
in the estimated impacts across key subsamples. In the final section, we 
summarize the main transfer-related findings from the demonstration, and 
highlight the main policy implications of these findings. 

A. PUBLIC TRANSFER DEPENDENCE IN THE ABSENCE OF STETS 

On the whole, control group members exhibited very little change 
over time in either the receipt or the amount of public transfers, as shown 
in the last column of Table VI. 1. This finding is generally counter to our 
expectations, since job-holding among controls increased by 28 percent over 
that period, and the proportion of the employed who held regular jobs 
increased from 18 percent at baseline to 42 percent at month 22 (see Table 
IV. 1 in Chapter IV). Even with reporting and administrative lags, our 
hypotheses would tend to suggest a decline in transfer dependence. 

The trends are not necessarily consistent across types of 

transfers. Virtually no change occurred between baseline and months 15 and 

22 in terms of the percentage who were receiving any cash transfers and the 

2 

average monthly income from cash transfers. However, both the percentage 



1 

One possible explanation for the observed results is an 
independent trend in increased SSI dependence as members of this target 
population aged. 

2 

The trend often diverges for transfer use and benefit amounts 
measured at month 6, but this pattern is likely to be an artifact of the 
small sample size for month 6, rather than any real divergence from the 
longer-term trend. 




TABLE VI. 1 





PUBLIC TRANSFER OCPEMOeNCC OF CONTROL GROUP HEHBERS 


, BY SITE 












Site 








Type And 
TlM Period 


Cincinnati 


Loo 
Angolos 


New 

York 


St. 
Paul 


Tucson 


Tot el 


^u«»Nt Rmiving Any 

8ml ln<* 

Month 15 
Month 22 


4t.9 
34.5 
55.8 
59.0 


51.2 
29.6 
46.2 
48.6 


56.9 
43.2 
40.4 
41.3 


44.4 

41.2 
48.1 
44.0 


69.6 
54.8 
65.9 
65.9 


53.8 
41.1 
5K5 
52.4 


%fifi|i NoNtldy XfieoM 

8mlino* 

Month 15 
Month 22 


54.65 
57.00 
103.55 
110.74 


162.10 
110.70 
173.63 
195.68 


123.70 
76.17 
107.21 
103.82 


138.62 
98.24 

149.35 
79.39 


154.38 
133.03 
158.30 
163.43 


125.16 
94.40 
136.46 
133.44 


^roM«t Kmiving 5S 
«r S9DI . 
BoMline' 

nDntn o 

Month 15 
Month 22 


20.9 
20.7 
23.3 
28.2 


31.7 
25.9 
43.6 
43.2 


27.5 
24.3 
31.9 
32.6 


25.9 
11.8 
33.3 
20.0 


$8.7 
54.8 
63.6 
63.6 


33.7 
29.1 
39.5 
39.3 


NtMtldy InooM 

Bmllno* 

Month 15 
Month 22 


36.96 
41.93 
55.27 
74.10 


142.63 
105.70 
173.34 
183.97 


65.63 
43.24 
86.21 
87.07 


33.30 
24.89 
52.42 
42.11 


122.87 
133.03 
145.93 
152.71 


83.12 
72.13 
104.80 
111.83 


PoraNt ImiviJig . 
Othir C«li TrM^n^ 
B«Mllne' 

Month 15 
Month 22 


20.9 
17.2 
37.2 
30.8 


22.0 
3.7 
2.6 
5.4 


31.4 
24.3 
8.5 
8.7 


37.0 
29.4 
29.6 
28.0 


17.4 
0.0 
6.8 
6.8 


25.0 
14.2 
16.0 
14.7 


Mfmgm flm/thXy IneoM ^ 
Frai Othor^CMli TrMfbcs^ 
Bmlino* 

rwnbn o 

Month 15 
Month 22 


16.84 
13.57 
48.27 


12.15 
5.00 
3.21 

11.40 


56.76 
31.72 
20.56 
16.39 


108.65 
71.88 
96.92 
35.65 


33.46 
0.00 
12.36 
10.73 


41.18 

20.71 
31.34 
20.80 


Foroowt Rmivlng 
Food St«M 

Bosellne 

Month 15 
Month 22 


25.6 
24.1 
27.9 
23.1 


9.8 
0.0 
12.8 
10.8 


37.3 
24.3 
25.5 
32.6 


3.7 
11.8 

7.4 
16.0 


30.4 
25.8 
15.9 
18.2 


23.6 
18.4 
19.0 
20.9 


PoraNt Using nriiem 
or ModlcAld 

BoMllne* 

Month 6 
Month 15 
Month 22 


11.6 
6.9 
34.9 
38.5 


46.3 
44.4 
48.7 
45.9 


56.9 
56.8 
55.3 
50.0 


51.9 
41.2 
44.4 

60.0 


17.4 
22.6 
47.7 
52.3 


36.1 
34.8 
46.5 

48.7 


^roont Liyliig in 
SiMdizod.HottslM 
BmoI 1 ne 

Month 6 
Month 15 
noncn cc 


11.6 
10.3 
9.3 
10.3 


9.8 
25.9 
15.4 
10.8 


2.0 
2.7 
4.3 
2.2 


37.0 
29.4 
29.6 

20.0 


10.9 
6.5 

13.6 
9.1 


12.0 
12.8 
13.0 
9.4 


Nuober in Siaplet 
Bwellne 
Month 6 
Month 15 


43 
29 
43 


41 

27 
39 


51 
37 
47 


27 
17 
27 


46 

31 
44 


208 
141 
200 


Month 22 


39 


37 


46 


25 


44 


191 



NOTEt These d«ta are unadjusted subgroup aeans. 



*In soae uses, baseline Intervlens were adalnlstered several weeks after randoa asslgruwnt. 

""other cash transfers primarily Include AFOC and general assistance. However, soee Individuals received dependent and survivor 
Social Security benefits, special state or local stipends for training or housing, and Unoiploytwnt Insurance benefits. 
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who were receiving SSI or SSDI and the average monthly income from these 
two sources show small upward trends as the sample aged. The percentage 
change for SSI/SSDI income is twice as large as the percentage change for 
those who were receiving such income, indicating that, over time, more 
persons received SSI or SSDI, and that, on average, recipients received 
higher benefits from these programs. The opposite is true for other cash 
transfers. Over time, both the percentage who were receiving other cash 
transfers and thf» average monthly income received by recipients fell. 
Virtually no change occurred over time in the percentage who were receiving 
food stamps or who were living in subsidized housing. The percentage who 
were using Medicare or Medicaid does show a trend of increased use. 

A few site patterns are worth noting. First, with the possible 
exception of the percentage who were receiving SSI or SSDI in St. Paul, a 
modest upward trend occurred in both the receipt and amount of SSI or SSDI 
in all sites as the sample aged. Second, the percentage who were ireceiving 
other cash transfers declined in all sites except Cincinnati. In St. Paul, 
reductions also occurred in the average monthly income from these other 
transfers — reductions that are proportionately larger than the reductions 
in receipt, indicating sizeable reductions in the average monthly amount 
per recipient. In Tucson and New York, the proportional reductions in 
average monthly income roughly match the reductions in receipt, while, in 
Los Angeles, virtually no reductions in the average monthly benefit amount 
accompany the large reductions in receipt. Thus, recipients in Tucson and 
New York were basically receiving the same amount over time, while 
recipients in Los Angeles were receiving increased amounts over time. The 
situation was very different in Cincinnati: the percentage who were 
receiving other cash transfers increased over time (by 47 percent between 
baseline and month 22), and the average rai)nthly income from those sources 



1 

Although the composition changed over time, most of the "other 
cash benefits" are welfare — either AFDC or general assistance. The 
remaining portion is accounted for by a variety of programs, including 
dependent and survivor Social Security benefits, special state or local 
stipends for training or housing, and, especially at months 15 and 22, 
Unemployment Insurance benefits. 
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increased by a greater amount (by 118 percent between baseline and month 
22). These trends for Cincinnati led to substantial increases in the 
benefit levels to recipients over time. 

Just a few site differences are associated with the receipt of in- 
kind transfers. Although Los Angeles and New York show no real trends, 
both Tucson and Cincinnati show large increases in the use of Medicare or 
Medicaid over time. The trend in Tucson may be associated with the fact 
that Arizona actually did not have a Medicaid program per se, but instead 
operated a state substitute which underwent changes during the course of 
the demonstration. Enrolling in the revised program appears to have been 
easier than enrolling in the program that existed at the start of the 
demonstration. Modest trends occurred in St. Paul for all three types of 
in-kind transfers, the largest of which was the increase in the percentage 
who were receiving food stamps. 

Table VI. 2 shows the attributes of control group members that are 
associated with their receiving selected types of transfers in month 22. 
Mere seriously disadvantaged sample members—as indicated by lower IQ 
scores, the lack of financial management skills, the presence of secondary 
handicaps, and organic causes of retardation—were more likely to be 
receiving SSI or SSDI and Medicare or Medicaid, and less likely to be 
receiving other cash transfers. Control group members who were male, 22 
years of age or older at baseline, or Hispanic or white were more likely to 
be receiving SSI or SSDI and Medicare or Medicaid than were others in the 
respective categories of subgroups. Sample members who were female, 22 
years of age or older, or black were more like to be receiving other "ash 
transfers. Baseline receipt of a certain type of transfer increased the 
likelihood of also receiving it at month 22. Finally, comparison group 
members who were living independently at baseline were much more likely to 
be receiving SSI or SSDI at month 22 than were those who were living in 
other arrangements, while those who were residing in supervised settings 
were more likely to be receiving other cash transfers and Medicare or 
Medicaid. 
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TABLE VI. 2 



PERCDfT OF CONTROL CROUP HEHBERS RECEIVING 
TRANSFERS IN MONTH 22, BY CHARAaERISTICS AT BASaiNE 



Subgroups Dtf Intd by 


Any 

Cash Transfers 


SSI/SSDI 


Other 
Cash Transfers^ 


Hedicare/Kedicaid 


Total Sm^ 


52.4 


39.3 


14.7 


46.7 


Sit* 

Cincinnati 
Lot Angtlcs 

nCW TOnC 

St. P«ul 
Tucton 


59.0 
46.6 
41.3 
44.0 
65.9 


26.2 
43.2 
32.6 
20.0 
63.6 


30.6 
5.4 
6.7 

26.0 
6.6 


36.5 
45.9 

50.0 
60.0 
52.3 


IQ UmI 

Hlld 
Hodentt 


43.3 
57,1 
52.6 


21.7 
46.2 
42.1 


21.7 
11.6 
10.5 


41.7 
53.6 
42.1 


Younger th4n 22 
22 or older 


45.7 
69.8 


32.6 
56.6 


13.6 
17.0 


43.5 
62.3 


FeMle 


53.6 
50.6 


41.5 
36.5 


12.3 
17.6 


50.0 
47.1 


RM/EtHnieity 
Black 
Hispanic 
ffhit* and other 


46.9 
46.3 
57.1 


25.0 
44.6 
46.9 


21.9 
6.9 
12.2 


35.9 
62.1 
D3.1 


UvlAf ArrwiiBirtt 

Living with Parents 

Living In supervised setting 

Living Independently 


47.9 
61.3 

72.7 


35.6 
50.0 
72.7 


13.5 
31.3 
9.1 


46.6 
75.0 
45.5 


FiMwdal ttinifeaefir Sklils 
Independent 
Not Independent 


45.7 
54.2 


26.3 
42.4 


19.6 
13.2 


37.0 
52.6 


SSI/SSDI 

Other transfers only 
No trsnsfers 


81.5 
46.5 
23.2 


72.3 
26.5 
16.1 


12.3 
23.5 
7.1 


70.6 
46.5 
23.2 


Secowdiry HWiesps 

Secondary hsndlcsp 
No secondsry handicap 


66.9 
44.6 


59.0 
30.0 


13.1 
15.4 


65.6 
40.6 


Cmm ef Itetardatlofi 

Organic 
Hon -organic 


62.8 
46.9 


62.6 
31.5 


3.4 
16.7 


55.2 
47.5 


Benafaetor 

Benefactor 
No benefactor 


51.7 
52.6 


43.1 
37.6 


12.1 
15.6 


53.4 
46.6 


Bock Exparianoa In Taa 
Years Priar ta CnrallMiit 
Regular Job lasting 

>3 Bonths 
Otlwr Job lasting 

>3 aonths 
Otlwr 


37.0 
46.3 
56.5 


22.2 
39.7 
43.4 


16.5 
6.6 
17.0 


33.3 
43.1 
55.7 


Sdiaol Status at Referral 

Enrolled 
Not enrolled 


62.5 
46.2 


47.2 
34.5 


15.3 
14.3 


51.4 
47.1 


Nuaber In Saaple 


100 


75 


26 


93 



NOTEt These data are unadjusted subgroup aeans. 

*0ther ca^h transfers prlnarlly Include AFDC and general assistance. However, some Individuals received dependent and survivor 
Social Security benefits, special state or local stipends for training or housing, and Uneaploynent Insurance benefits. 
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B. OVERALL PROGRAM EFFECTS 

In this section, we discuss the overall impacts of the program 
measured at three points in time. We first discuss the impacts at month 6, 
which we have characterized as the in-prograro period. We then discuss the 
impacts at months 15 and 22, which we have characterized as the postprogram 
period. 

1. In-Program Effects 

As described at the start of this chapter, the impacts of the STETS 
program on employment that were described in Chapter IV led to the 
expectation that program participation would also reduce dependence on cash 
transfers. However, we also expected that the reduction would occur with a 
lag, since both reporting and administrative responses would not likely 
have kept up with changing labor-market activities. In fact, as shown in 
Table VI. 3, the estimated experimental effects consistently show evidence 
of reduced dependence on all transfers other than food stamps. However, 
only the effects that were estimated for the percentage who were receiving 
any cash transfers, the percentage who were receiving cash transfers other 
than SSI and SSDI (primarily welfare), and the percentage who were living 
in subsidized housing are statistically significant. Experimentals were 26 
percent less likely to be receiving any cash transfers (32 versus 43 
percent), and were 58 percent less likely to be receiving cash transfers 
other than SSI and SSDI (6 versus 14 percent). A substantial effect also 
occurred in terms of the percentage who were living in subsidized housing— 
a reduction of about one-third (10 versus 15 percent). However, such an 
immediate effect on housing of this magnitude seems unusual, and this 
effect may well hav.-. been da to the small numbers of sample members who 
were living i.. -."--Idized housing (see Chapter VII, Section VII. B). 

2. Postprogram Effects 

The estimated experimental effects at month 15, by which time most 
experimentals were no longer in the program, followed the pattern esta- 
blished for month 6, but with some differences. The statistically sign- 
ificant effects estimated at month 6 for both the percentage who were 
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TAdl£ VI.3 

ESnHNTO) PROCaW IffAaS ON WBLIC nWNSFIR DEPOCOCE 



Moftth 6 Konth IS Montn 22 

Control Oontrol Control 

ExperiiMental Qroup Estimated Experlnental Qraup Estimated Experimental Occup Estimated 

Ootccwe Measures Qroup Mean Hean Impact Qroup Mean Mean Impact Group Mean Mean Impact 

FVroent Receiving Any 31.7 43.1 -11.4** H.5 51. S -7.0* 49.6 S.O -ZA 

Cash Transfers 

Average MLhXy Income froii $80.23 $99.98 $-19.75 $m.78 $136.72 $-23.94 $125.53 $136.06 $-9.55 
Cash Transfers 

Percent Receiving SSC 26.3 31.0 -4.7 33.1 40.7 -7.6»* 34.9 40.2 -5.3 

or SSI 

Average Itonthly Inco«e $66.41 $74.59 $-8.18 $91.35 $109.65 $-18.» $99.27 $120.03 $-20.76 
from SSI or SSI 

Percent Receiving 0tl«r 6.0 14.3 -«.3*» 12.4 14.7 -2.3 18.0 13.4 4.6 

Cash Transfers^ 

Average Monthly Inccne $13.04 $22.42 $-9.38 $22.26 $29.71 $-7.45 $29.23 $19.45 $9.78 
fropi Other Transfers* 

Percent Receiving Food 22.0 18.4 3.6 22«1 18.0 4.1 21.8 19.8 2.0 

Stamps 

Percent Using Medicare or 29.7 35.9 -6.2 41.0 46.4 -5.4 45.9 48.1 -2.2 

Medicaid 

fVrcent Living in SiAsidized 9.9 15.3 -5.4» 13.7 13.9 -0.2 11.0 10.0 1.0 

Housing 

tMer in Soiple S7 413 398 

Percent of Experimental s 67.8 18.3 0.0 
in niase 1 and 2 

NOTE: These results were estimated through ordinary least squares techniques. Definitions and means of control variables that are included in 

the modexs are presented In Appendix Table A*1. Full results frca a representative set of the liipact equations that uxlerlle these 
results are presented In Appendix Tables A.9 and A«10. 

^her cash trxisfers pi'lmarlly include AFDC and general assistance. However, some individuals received dependent and survivor Social Security 
benefits, special state or local stipends for training or housing, and Uhei^ployment Insurance benefits. 

•Statistically slgilflcwit at the 10 percent level, two-tailed test. 
**Statlstically significant at the 5 percent level, two-tailed test. 
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receiving cash transfers other than SSI and SSDI and the percentage who 
were living in subsidized housing did not persist into month 15. In fact, 
for the housing outcome measure, virtually no program impact existed by 
month 15. Instead, the effect estimated for the percentage who were 
receiving SSI or SSDI is sizeable and statistically significant. 
Experimentals were 19 percent less likely than were controls to be 
receiving such transfers at that time (33 versus 41 percent). Due largely 
to this result, experimentals were also 14 percent less likely to be 
receiving any cash transfers (45 versus 52 percent). 

By month 22, no estimated effects were statistically significant, 
and the direction of the point-estimates of the effects was not even 
consistently negative. However, it may be noteworthy that the estimated 
effect on the receipt of SSI or SSDI was still negative and relatively 
large, indicating that experimentals were 13 percent less likely to be 
receiving such benefits than were controls (35 versus 40 percent). 

A further examination of these patterns of program-induced changes 
in public transfer dependence in the postprogram period revealed that the 
program impacts on the probability of receiving such transfers are directly 
related to changes in the incidence of employment. The average impacts on 
overall benefit levels offset less than 20 percent of the earnings gain, 
while the effect on SSI and SSDI offset over one-third of the earnings 
gain. rhus, it is clear that STETS not only reduced public transfers, but 
also tended to move some individuals off SSI and SSDI onto other forms of 
assistance. 

C. PROGRAM EFFECTS FOR SELECTED SUBGROUPS 

In view of the limited effects of STETS on the entire sample, it is 
appropriate to estimate some sets of program effects by subgroups to iden- 
tify patterns that may be hidden in the more aggregate analysis. The out- 
come measures for the subgroup analysis—average monthly income from SSI or 
SSDI and average monthly income from other cash transfers— were selected 
because they were deemed to reflect changes in both the percentage of the 
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sample who were receiving the respective types of transfers and the average 
monthly income of those who were receiving benefits. 

As shown in Tables VI.4.A and VI.4.B, no clear and persistent 
patterns of site differences emerge for either outcome measure. At any 
point in time, the experimental effects estimated for selected sites do 
stand out at one extreme or another. However, the pattern of effects 
changed dramatically over time in ways that preclude us from drawing any 
conclusions about site patterns. 

Some patterns do emerge among other subgroups. Of course, given 
the modest sample sizes and the general lack of significant experimental 
effects even for the overall sample, all subgroup patterns must be regarded 
as very speculative. Among subgroups defined by IQ level, the estimated 
effects for those with a mild level of retardation were consistently 
negative for SSI/SSDI benefits, grew in absolute value over time, and 
became statistically significant even for the modest sample size. 
Reductions in other cash benefits were also estimated for months 6 and 15, 
but, for month 22, it is estimated that the program increased the receipt 
of other cash benefits. On the other hand, the effects estimated^ for those 
with moderate retardation were consistently positive for both outcome 
raeasuies, and were quite large by month 22, 

Estimated reductions in SSI and SSDI benefits were particularly 
large and consistent over time for males and blacks. Reductions estimated 
for younger sample members were also consistent over time and, in month 22, 
were larger than those that were estimated for older sample members. 
Reductions that are large relative to others in their respective subgroups 
were estimated for months 15 and 22 both for sample members who were living 
independently and for those with an organic cause of retardation. For some 
other sets of subgroups, particularly those defined by IQ level and 
financial management skills, some very large and occasionally significant 
estimated effects occurred, but without clear patterns. 



1 

The full regression results for the models from which these two 
sets of results were generated are shown in Appendix Tables A.9 and A. 10. 
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TABL£ VIA 



ESnmTH) PROGRAH I^ACTS Oi INCOtC HCH TRANSFERS FtXt 
KEY SUBGROUPS OF STETS PWTICIP/WS 

A* AVERAGE HOKDIY INCO« HCH SSI OR SSDI 







Month 6 






Month 15 




Month 22 




SU]graups Def Ifwd 
by Characteristics 
«t BMllne 


ExperUiental 
Qroup Nean 


Control 
Mean 


EstlMeted 
Impact 


Experimental 
Qroup Mean 


Control 
Me«i 


Estimated 
Invact 


Experimental 
Qroup Hem 


Control 
Me«i 


Estimated 
Impact 










$91.35 


$109.65 


$-18.X 


$99.27 


$120.03 


$-20.76 


Site 

ClndnrMtl 
Los Angeles 
York 

St. p«a 

Tucson 


56.11 
t06.C3 
9.5% 
12.%8 
76.14 


64.64 

119X)2 
49.23 
40.91 
91.74 


-6.53 
-12.39 

10.31 
-26.43 
-15.60 


71.64 
180.10 
<)6.56 
13.15 
^.92 


78.06 
186.19 
119.02 
57.24 
66.44 


-6.44 
-6.09 
-32.46 
-44.09 
-11.52 


67.34 
180.56 
85.42 
49.65 
93.74 


105.07 
207.39 

ias.li 

X.69 
96.90 


-37.73 
-26.81 
-40.69 
18.96 
-3.16 


Borderline 

NUd 

Moderate 


56.49 
64.39 
96.17 


72.54 
72.06 
92.07 


-14.05 
-7.69 
4.10 


82.15 
90.55 
122.54 


81.92 
121.63 
119.68 


0.23 
-31.06* 
2.66 


89.70 
98.02 
131.85 


63.36 
143.75 
85.16 


6.32 
-45.73»» 
46.69 


Yomger than 22 
22 or oldsr 


61.34 
80.46 


81.25 
56.09 


-19.91 
24.39 


64.05 
106.06 


100.76 


-16.71 
-21.95 


85.92 
129.21 


112.09 
137.66 


-26.17 
-8.65 


Qender 
Male 
FeMie 


72.83 
56.09 


71.50 
78.9 


1.33 
-a}.50 


81.86 
83.80 


121.16 
94.02 


-39.30»» 
-10.22^ 


91.46 

i09.as 


133.93 
102.24 


-42.47»» 
7.02 


Rne/Etiwdelty 

Black 
Hispanic 
Ihlte «d other 


40.82 
47.61 
0^.55 


60.31 
62.47 
65.03 


-19.49 
-14.86 
*0.48 


52.66 
106.82 
1V.29 


108.18 
81.69 
118.10 


-55.32»* 
25.13 
-7.81 


67.x 
94.36 
118.04 


92.x 
94.56 
141.68 


-25.20 
-o!22 
-23.64 


Uving AmngHvt 
Living with parents 
Living In supervised 

setting 
Living Independently 


64.80 
26.24 

136.25 


74.46 
77.81 

71.91 


-9.66 
-51.57 

64.34 


87.12 
102.65 

123.89 


103.54 
121.92 

161.12 


-16.42 
-19.27 

-37.23 


102.13 
78.11 

96.81 


111.06 
102.77 

229.20 


-8.95 
-24.66 

-132.39** 


HiMRBial nii^Mrt SkUU 
xnaepenoenC 
Not Independent 


66.» 
66.46 


80.21 
72.33 


-13.91 
-5.87 


89.81 
91.92 


92.18 
116.20 


-2.37 
-24.28* 


92.74 
101.73 


122.46 
119.12 


-29.72 
-17.39 


RBoe|pC of Tramftan 

SSl/SSDI 

Other transfiers only 
No transfsrs 


175.17 
21.21 
8.06 


194.81 
18.46 
14.74 


-19.64 
2.75 
-6.66 


199.01 
37.33 
3S.52 


202.24 
75.83 
50.63 


-3.23 
-36.50* 
-14.11 


203.06 
53.91 
36.12 


180.55 
114.13 
63.66 


22.51 
-27.54 


Cm ofRiUvdatkn 

Organic 
Non-«rganlc 


90.44 
61.79 


31.99 
82.79 


5e.45» 
-21.00 


100.70 
89.53 


165.43 
98.82 


-64.73»» 
-9.29 


92.91 
V0.56 


186.57 
106.52 


-93.66»* 
-5.96 


Mc CMimlenue in Tao 
Vntb PHor to Enralkmfc 
Regular job lasting 

>3 months 
O^r >>b lasting 

>3 Months 
OtHir 


9.23 
76.62 
61.35 


75.84 
84.25 
67.45 


-16.61 
-7.63 
-6.10 


1M.99 
79.68 
88.13 


103.28 
102.22 
116.x 


27.71 
-22.54 
-28.17 


96.19 
106.51 
95.34 


141.25 
87.67 
135.51 


-45.06 
18.64 
-40.17** 


Scteol SUbM «tllBreRial 

Enroll 
Not enrolled 


71.70 
64.04 


72.24 
75.65 


-0.54 
-11.61 


im.98 
80.39 


124.23 
102.89 


-9.25 
-22.50 


116.55 
91.36 


132.02 
114.55 


-15.47 
-23.19 


Hjito In Saiple 






287 






406 






399 



NOTEt These reults were estlmted through ordinary least sqi^^es techniques. In addition to the control verifies that were iwluded In the 
models which underlie the werall net imfM estl«at49S reported In Table VI .3, ttiese «)dels Inclutte yxid>l» that Interacted the 
treatment variAle with the subgroup verifies, full results ftom the rosresslon model for the month 6 «id mtth 22 outocmes »e 
presorted In Appendix T^les A.9 ^ A. 10. 

■Statistically significant at the X percent level, twc-talled test. 
••Statistically significant at the 5 percent level, two- tilled test. 
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T/«t£ VI.^ 



Es^H^TtD Pfwawi wAas incoc proh transfeks for 

KEY SUBOCUPS OF STHS P^HCIPAKTS 
B. AVER/WE MONTHLY INCWC FTCH OT>€R CA* TRANSFERS* 







Month 6 






Month 15 




Month 22 




t>y Chtfacterlstics 
«t B«6llne 


ExperiMental 
Ctouo Hean 


Control 
Qroup 
Mean 


Estimated 
Impact 


Experimental 
Ctouo Mean 


Control 
Qroup 
Mean 


Estimated 
Imapct 


Experimental 
Grouo Mean 


Control 
Qroup 
Mean 


Estimated 
Impact 




$13.0% 


$22.%2 


$-9.38 


$22.26 


$29.71 


$-7.%5 


$29.23 


$19.%5 


$9.76 


SttB 

Cincinnati 
Los Angeles 
New York 
St. Pail 
Tucson 


2.99 
OM 
19.80 
73.00 
(6.2%) 


21.50 
0.11 
36.68 
67.96 
2.87 


-18.51 
0.31 

-16.88 
5.0% 
-9.11 


1%.%6 
5.11 

as.63 

82.61 
6.77 


%5.3% 
%.78 
18.01 
89.76 
17.%0 


-30.86 
0.33 
8.62 
-7.15 

-10.63 


W.11 
19.50 
25.95 
62.6% 
13.16 


27.06 
7.01 
10.70 
%0.06 
21.71 


13.03 
12.%9 
15.25 
22.56 
-6.55 


Borderline 

HUd 

Moderate 


8.86 
12.70 

a%.77 


15.1% 
26.81 
19.52 


-6.28 
-1%.11 
5.25 


30.28 
1%^ 
%%.76 


25.37 
31.85 
ZJ.66 


%.91 
-17.63 
15.10 


2B.59 
27.58 
39.73 


23.20 
20.17 
6.17 


5.39 
7.%1 
33.56 


AOB 

Youiger than 22 

22 or older 


16.32 
%^1 


16.37 
3S.68 


-0.05 
-3%.%7»» 


27.07 
11. %1 


26.58 
36.76 


0.%9 
-25.35 


>^.38 
17.99 


12.09 
35.53 


22.29^ 
-17.5% 


(Msr 

Hale 
FeMle 


7.89 
19.77 


27.8% 
15.32 


-19.95* 
%.%5 


7.75 
%2^1 


26.62 
33.96 


-18.87 
8.25 


19.73 
%1.7% 


1%.01 
26.62 


5.72 
15.12 


(UMAEthOcity 

auck 

Htspvile 
Wilts «vl other 


6.27 
1%.57 
16.55 


15.92 
17.19 
27.07 


-9.65 
-2.62 
-10.52 


35.%3 
19.58 
15.1% 


2D.X 
38.62 
32.89 


15.13 
-19^ 
-17.75 


51.85 
12.92 
20.70 


3%. 91 
20.77 
10.%0 


16.9% 
-7.65 
10.30 


Uvlfn Arx«9B»irtt 

Living with parents 
Living in supervised 


13.68 
10.88 


21.87 
%7.7% 


-6.19 
-36.86 


2S.%% 
(8.56) 


32.25 
3%.61 


-5.81 
-%3.17 


29.12 
22.25 


18.75 
%1.16 


10.37 
-16.91 


setting 
Living independeftly 


8.%% 


^.80 


11.2% 


15.59 


-2.25 


17.8% 


38.65 


-0.0% 


38.69 


FiiMwelM Hwipwit SkUls 

Independent 
Not independent 


1%.18 
12.59 


%1.80 
1%.78 


7.62» 
-2.19 


25.52 
21.06 


50.17 
22.17 


-2%.65 
-1.11 


<l0.31 
25.06 


%2.58 
10.79 


-2.27 
1%.29 


MneVt or TrMftai 

SSI/SSDI 

Other transfers only 
No transfers 


15.06 
29.%3 
(1.11) 


13.96 
62.71 
-0.75 


1.10 
-33.28*» 
^.36 


35.07 
2D.%% 
11.%6 


25.53 
52.%0 
12.03 


9.5% 
-31.96* 
-0.57 


29.55 
27.2% 


13.65 
37.W 
7.77 


15.70 
-6.%6 
19.%7 


Cane of RiUnUUon 

Organic 
Non-organic 


15.%2 
12.56 


2%. 73 
21.96 


-9.31 
-9.W 


6.21 
25.%9 


1%.65 
32.75 


-6.%% 
-7.26 


2%.29 
X.26 


10.21 
21.38 


1%.08 
6.88 


Vbric &pKlQnD8 In Tvo 
Vurs PHor to Cmlta* 

Regular Job lasting 

>3 Months 
OtJvr job lasting 

>3 Months 
OtRr 


(3.76) 
21.17 
12.15 


20.06 
22.2% 
23.25 


-23.6% 
-1.07 
-11.10 


7.7% 
39.86 
1%.50 


X}.88 
9.2% 
31.96 


-63.1%»* 
».62* 
-17.%6 


12.59 
46.71 
22,05 


%2.16 
10.03 
19.%2 


-29.57 
36.68»* 
2.63 


School SUto ait Refeml 

Enrolled 
Not enrolled 


10.18 
1%.32 


32.63 
17.7% 


-22.65 
-3.%2 


13.31 
26.3% 


36.61 
26.%6 


^3.50 
-0.1% 


3^.%6 
26.63 


29.32 
1%.92 


5.1% 
11.91 



NMber in Sa^le 



28% 



%13 



%01 



NOTEt These results were estimated trrough ordinary least squares techniques. In addition to the control vari*les that arc Included In the 
«dels which inlerlle the overall net Ijpact estimates r«ported in T*le VI.3, these models Include vari^les that Interacted the 
treatiiont variAle with the subgroup variAles. Full results fwa the recession models for the month 6 and month 22 outccmes arc 
presented in Appendix TAles A. 9 and A* 10. 

*Other cash tp*«fters primrily Includfe AfDC vti general assistm. Hoiwvcr, some Individuals received dependent and survivor Social Security 
benefits » special state or local stipends for training or housing, and Uhonployment Insurance benefits. 



•Statistically significant at the 10 percent level » two-tailed test. 
•♦Statistically significant at the 5 percent level, two-tailed test. 
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The trends are quite different for cash transfers other than SSI 
and SSDI: the effects estimated for older sample members and Hispanics 
were ^re consistently negative and generally larger than those estimated 
for ocners in the respective sets of subgroups. Females, sample members 
who were living independently, and, after month 6, blacks showed evidence 
that their receipt of these other cash transfers increased — patterns that 
sometimes, but not always, parallel the earnings results for these 
subgroups, reported in Appendix Table A.4.B. 

D. SUMMARY AND CONCLUSIONS 

The evaluation of the impacts of STETS on cash and in-kind transfer 
dependence began with a strong hypothesis—that, conditional on the impacts 
of the program on earnings, STETS participation would reduce the receipt 
and, for cash transfers, the amount of transfers. While impacts on 
transfers were expected to lag behind impacts on earnings, it seemed likely 
that they would have occurred by the end of the observation period, given 
the pattern of earnings Impacts. However, the results generally did not 
show this expected pattern of effects. In fact, the impacts that were 
observed occurred early and then tended to fade. The one encouraging sign 
is that the pattern for SSI and SSDI — point-estimates of reductions in both 
the receipt and amount of SSI or SSDI — persisted over time. In month 22, 
these reductions offset 38 percent of the total earnings gain. While the 
estimated effects are generally not statistically significant at the 
conventionally accepted levels, the point estimates are reasonably large. 
We believe that a useful practice would be to follow these SSI/SSDI-related 
outcomes for a longer period of time, perhaps by using Social Security 
records data. 
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VII. IMPACTS ON ECONOMIC STATUS, INDEPENDENCE, 
AND LIFE-STYLE 



As was discussed in Chapter IV, the STETS program had a substantial 
impact on the incidence of regular job-holding and on the work hours in and 
earnings from those jobs. It might be expected that the STETS demonstra- 
tion would also have impacts on other areas of participants' lives— espe- 
cially their overall economic status, their independence in financial man- 
agement and living arrangement, their use of formal and informal services, 
and their general level of involvement in regular, productive activities. 

However, the expected direction and duration of the effects of the 
program on economic status, measures of independence, and life-style are 
not always clear. Several factors in particular cloud the results of any 
present evaluation of these impacts. First, the increased earnings 
observed for experimentals appear to be offset partially by decreases in 
transfer benefits and other sources of income, thereby diluting the overall 
financial impacts of the program. Second, although STETS may have had 
impacts on financial management skills and independent living arrangements, 
those impacts may follow others with a considerable time delay, in which 
case the 22-Tnonth observation period of our study may be too short to 
observe such program impacts. Third, although the program generated 
increased earnings for sample members, those increases might not have been 
enough to anable them to live independent life-styles — especially in such 
large metropolitan areas as N^w York or Los Angeles. Finally, parents and 
counselors might simply wish to see more concrete and stable earnings gains 
before they are willing to give the sample members greater independence. 

Despite these limitations in our ability to detect what may be 
primarily long-run effects, we have pursued an analysis of such impacts due 
to the strong policy interest in understanding the effects of transitional- 
employment programs such as STETS on the life-styles of participants, in 
addition to or because of its success in integrating mentally retarded 
individuals into the workplace. To varying levels of detail, as permitted 
by the availability and quality of available data, this chapter explores 
the impacts of STETS on the following outcomes: 
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• Personal income from all sources 

• Independence In financial management 
o Use of formal services 

• Existence of a personal relationship vd.th a 
benefactor 

• Involvement in some regular activity which could 
lead to integration into the labor market 

• Living arrangement 

• Family status 

As in the previous chapters, we begin by providing an overview of 
the experience of the control group to describe how STETS participants 
would have fared in the absence of the demonstration. In Section B, we 
discuss the impacts of the program on specific outcome measures. In the 
final section, we summarize the main findings and discuss their policy 
implications. 

A. EXPERIENCE IN THE ABSENCE OF STETS 

The experience of control group members in texrms of economic 
status, independence, and life-style reflect the expected experience of the 
experimental group had they not enrolled in the STETS program. 

!• Economic Status 

Weekly personal income was measured as the sum of all earnings, 
transfer program benefits, and any other regular sources of income per 
week. As shown in Table VII. 1, average weekly personal income from all 
sources varied considerably across sites (from $27 in Cincinnati to $57 in 
Los Angeles at baseline, and from $29 in Cincinnati to $79 in New York at 
month 22). However, even at its highest level, the average total income of 
the STETS control group ($66 per week) was only about 60 percent of the 
federal poverty-income guidelines for family units of one, which were 
$77.11 per week at the start of the demonstration, 1982, $93.46 in 1983, 
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TABLE VII. 1 



WEEKLY PERSONAL INCOME FROM ALL SOURCES FOR 
CONTROL f • •iP MEMBERS, BY SITE 



Site 

Time Period Cincinnati Los Angeles Wem York St. Paul Tucson Total 

Total Dollars of 
Personal Incoae 

Baseline^ $26.61 $57.28 $^1.52 $^3.6^^ $^9.i^3 $^f3.5^ 

Month 6 31.53 43.59 65.^^3 48.63 55.90 49.63 

Month 15 30.64 53.88 86.32 65.12 56.74 58.24 

Month 22 28.75 56.81 79.27 63.75 71.44 60.69 



Number In Sample: 

Baseline 39 36 48 26 44 193 

Month 6 28 27 32 17 29 133 

Month 15 44 38 45 26 42 195 

Month 22 39 36 45 25 44 189 



NOTE: Sources for total personal Income Include earnings from Jobs, transfer-program (SSI, SSOI, 
and welfare) benefits. Social Security benefits, stipends or grants, state supplements for 
residential support, and regular contributions by parents, based upon reports by primary 
sample members or proxies. 

^In some cases, the baseline Interviews were administered several weeks after random assignment. 
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and $97.77 at the end of the follow-up period, 1984. Had these young 
people been living on their own (which, by and large, they were not), their 
general econooic status would have been very low. 

In general, average personal income among the entire control group 
did increase substantially over time, from $44 per week at baseline to $61 
per week at month 22 (39 percent). However, all of this increase is 
attributable to increases among the New York, St. Paul, and Tucson 
samples — 90^, 46, and 45 percent, respectively. In Cincinnati, total 
personal income remained low throughout the demonstration ($27 to $31 per 
week), and although the control group in Los Angeles had the highest 
average personal Income at the start of the demonstration ($57 per week) 
they experienced virtually no increase over the 22-month follow-up period. 

2. Independence and Life-Style 

Independence and life-style encompass a variety of concepts, only a 
tew of which could be analyzed within the context of this evaluation. Tlie 
specific concepts considered below include financial management skills, the 
use of support services, personal relationships with benefactors, the 
extent of inactivity, and living arrangement. 

Financial Management Skills . The interviews provided data on 
whether sample members received assistance from anyone in three areas of 
financial management — paying sales clerks when shopping, handling bills 
(i.e., arranging for the payment of but not necessarily providing the funds 
for bills), and transferring money in or out of bank accounts. Those who 
performed at least two of the three activities without assistance (and 
received no assistance on the third) were considered to exhibit financial 
raanagemert skills. As shown in Table VII. 2, between 25 and 35 percent of 
the contr->l group members exhibited financial management skills at each 



1 

These figures are reported in the following three respective 
citations of the Federal Register: Vol. 49, No. 39, February 27, 1984, p. 
15418; Vol. 48, No. 34, February 17, 1983, p. 7010; and Vol. 49, No. 39, 
February 27, 1984, p. 7152. 
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TABLE VI 1.2 



SOCIAL SUPPORT Ah© INDEPENDENCE OF 
CONTROL GROUP MEMBERS, BY SITE 



Type and 
Time Period 



Site 



Percent Who Demonstrate 
Ind^enden^ In Finmial 
Hanaycinty^ 

Baseline 25.6 

Month 6 27.6 

Month 15 25.0 

Month 22 28.2 

Peroent Receiving Any 
Seivice ttm Agency^ 

Baseline 

Month 6 65,5 

Month 15 68.2 

Month 22 55.7 

Percent Receiving 3ob 

Training Seivices 

Baseline 1^.7 

Month 6 20.7 

Month 15 25.0 

Month 22 23.7 

Percent Receiving Job 

Search Assistance 

Baseline 29.5 

Month 6 17,2 

Month 15 15,9 

Month 22 17,9 

Percent Receiving School 
Services 

Baseline (fg.s 

Month 6 17.2 

Month 15 9.1 

Month 22 7,7 

Percent Receiving 

Residential Counseling 

Baseline 6,8 

Month 6 6,9 

Month 15 (j,5 

Month 22 7,7 

Percent Receiving Other 
Counseling 

Baseline (^5,5 

Month 6 3i^.5 

Month 15 5i^.5 

Month 22 51.3 

Percent Receiving 
Assistance with 
Financial Hanageaent^ 

Baseline 7,0 

Month 6 6,9 



Cincinnati Los Angeles New York St. Pau 1 



Tucson 



Total 



26.2 
22.2 
33.3 
31.6 



92.9 
92.6 
69.2 
73.7 



'fO.5 

28.2 
26.3 



35.7 
23.1 
^5A 
13.2 



16.7 
29.6 
7.7 



9.5 
22.2 
10.3 
13.2 



69.0 
66.7 
51.3 
60.5 



U.3 
22.2 



35.3 
27.0 
'fl.7 
^7.8 



88.2 
73.0 



^^.9 

29.2 
21.7 



^3.1 
22.2 
8.3 
15.2 



37.3 
21.6 
14.6 
8.7 



0.0 
0.0 
0.0 
0.0 



56.9 
54.1 
35.4 
37.0 



2.0 
8.1 



18.5 
38.9 
25.9 
25.9 



92.6 
83.3 
85.2 
74.1 



14.8 
50.0 
46.2 
42.3 



55.6 
17.6 
11.1 
11.1 



11.1 
11.1 
0.0 
3.7 



29.6 
27.8 
37.0 
25.9 



63.0 
55.6 
63.0 
51.9 



22.2 
11.1 



17.0 
32.3 
40.9 
34.1 



93.6 
80.6 
72.7 
72.7 



27.9 
38.7 
27.3 
38.6 



30.4 
25.8 
20.5 
11.6 



51.1 
12.9 
18.2 
20.5 



10.6 
6.5 

11.4 
6.8 



68.1 
61.3 
54.5 
63.6 



17.0 
16.1 



25.2 
28.9 
34.2 
34.5 



90.0 
78.2 
68.3 
65.5 



30.5 
39.0 
29.9 
29.7 



37.6 
21.6 
14.4 
14.0 



34.8 
19.0 
10.9 
12.4 



9.5 
10.6 
10.4 

9.3 



60.2 
54.2 
50.5 
52.6 



11.4 
12.7 
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TABLE VI 1. 2 (continued) 



Type anci 






Site 








Time Period 


Cincinnati 


Los Angeles 


New York 


St. Paul 


Tucson 


Total 


Month 15 


6.8 


7.7 


2.1 


29.6 


9.1 


9.4 


Month 22 


5.1 


18.4 


2.2 


29,6 


11.4 


11.9 


































7 1 




>•/ 


10.6 


9.0 


Month 6 


10.3 


7.4 


0.0 


1*5.7 


9.7 


7.7 


Month 15 


1 ^. o 


7 7 
f • f 


*f 




O.O 


7 h 


Month 22 


20.5 


2.6 


4.3 


3.7 


9.1 


8.2 


neroenv wiin ncnci ocxor 














Ddscii ne 


1^.6 


5Q ^ 


27. 5 


40.7 


46.8 


30.8 




17 5 




21.6 


16.7 


22.6 


24.6 


Month 15 


13.6 


23. 1 


o« 1./ 








Month 22 


10.3 


13.2 


13.0 


25.9 


27.3 


17.5 


Number In Sample^ 














Baseline 




42 


51 


27 


47 


211 


Month 6 


29 


27 


37 


18 


31 


142 


Month 15 


44 


39 


48 


27 


44 


202 


Month 22 


39 


38 


46 


27 


44 


194 



^Independence is defined as performing without assistance at least two financial management 
activities (shopping, handling bills, and using bank accounts), and without assistance In the 
other financial management activity. 

^In some cases, baseline interviews were administered several weeks after random assignment. 

^The services that were provided by agency staff members Included job training. Job-search 
assistance, schooling, residential counseling, other counseling, financial managemsr.t 
(assistance with shopping, bill paying, or handling bank accounts, or receiving transfer-program 
benefits on behalf of the individuals), and transportation (providing regular transportation to 
Job, training program, or school). 

^Thls heading is defined as assistance with shopping, handling bills, using bank accounts, or 
receiving transfer-program benefits on behalf of the individual. 

This heading is defined as providing regular transportation to a Job, training program, or 
school. 

f 

Benefactors are individuals named by primary sample members as providing assistance in two or 
more of the following areas* Job search, residential counseling, other counseling, financial 
management, and transportation. These individuals could be relatives or friends of the sample 
member, or service agency staff members. 

^Sample sizes for individual outcomes varied due to missing data. The numbers for the total 
sample ranged from 203 to 211 at baseline, from 141 to 142 at month 6, from 201 to 202 at month 
15, and from 192 to 194 at month 22. 
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interview wave. Among those who did not, most either had only one 
financial management opportunity which they performed independently or had 
opportunities in two or more financial management activities but were 
dependent in at least one activity; very few exhibited total dependence in 
financial management or had no opportunities to exercise financial manage- 
ment skills. 

Over the 22-month period, controls in Cincinnati experienced the 
smallest gains in the percentage who exhibited financial management skills 
(3 percentage points), while those in Tucson experienced the largest gains 
(17 percentage points). Only in New York did almost half of the control 
group members demonstrate financial management skills by month 22. These 
relatively low levels of independence may be attributable in part to the 
low levels of income available to the individuals and/or to the fact that 
most continued to live with their parents or other responsible adults. 

Service Receipt . The control group had access to and received a 
variety of services throughout the demonstration period. As shown in Table 
VII. 2, and as was expected given that most sample members were referred to 
the program through the service network, service receipt was particularly 
high at baseline (90 percent), although service use declined over the 
follow-up period. (In month 22, 65 percent of the j;ample still maintained 
contact with a service agency.) By far the most commonly used type of 
service was counseling outside of residential settings, which was used by 
50 to 60 percent of the control group at each time period. 

As was --.ted in Chapter V, about 30 percent of the control group 
received job training. Although variations occurred across the five sites 
in the use of job-training, the overall average level of use remained 
generally the same f:hrough month 22. While about one-third of the controls 
also used iob-search assistance and schooling services at baseline, the 
extent to which they used these services tended to drop off fairly quickly, 
to less than 15 percent by month 22. 

Only a small proportion of the control group (9 to 13 percent) used 
the remaining types of services — residential counseling, assistance with 
financial management, and transportation assistance. However, great 
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variations occurred across the sites in the level of such use. For 
example, generally over 25 percent of the control group in St. Paul 
received residential counseling anH a3sistance with financial management, 
compared with the overall sample average of approximately 10 percent. The 
control group in Cincinnati was twice as likely as the entire sample to 
receive transportation services at baseline and at month 22. These 
variations reflect differences in the types of support services available 
in the demonstration communities, as well as differences in individual 
needs for particular services. 

Personal Relationships with Benefactors , A much smaller proportion 
of the control group (31 percent) had a personal relationship with a bene- 
factor at the time they were referred to STETS. ^' Control group members in 
St. Paul and Tucson were the most likely to have had a personal relation- 
ship with a benefactor (41 and 47 percent, respectively), while those in 
Cincinnati were the least likely to have had one (14 percent). In all 
sites, personal relationships with benefactors waned over time; only 18 
percent reported having such relationships by month 22. 

Degree of Inactivity. As we discussed in Chapter IV, most control 
group members were involved in some activity pertaining to employment or 
training. As shown in Table VII. 3, nearly 40 percent were inactive in each 
time period— that is, they were not employed at least 4 hours a week and 
were not enrolled in a training or school program. In general, the 
activity status of the control group did not change substantially over 
time—with the exception of the control group in St. Paul (where the 
percentage who were inactive decreased by 63 percent) and the control group 
in Cincinnati (where percentage who were Inactive increased by 79 
percent). The substantial proportions of controls who were inactive 
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Benefactors included relatives, friends, or agency staff members 
who provided assistance in two or more of the following key areas: job 
search, residential counseling, other counseling, financial management, and 
transportation. 

2 

The decrease in St. Paul appears to have been due to the greater 
number who participated in training (particularly .n workshops and activity 
centers), while the increase in Cincinnati appears to have been due to the 
greater number who left schooling programs (see Chapters IV and V). 
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TABLE VII. 3 



PERCENT OF CX)NTROu GROUP MEMBERS WHO WERE INACTIVE, BY SITE 



Site 



Time Period 


Cincinnati 


Los Anqeles 


New York 


St. Paul 


Tucson 


Total 


Percent Inactive^ 














Baseline" 


37.2 


«.2 


35.3 


59.3 


23.9 


38.3 


Month 6 


55.2 


29.6 


21.6 


:J3.3 


35.5 


34.5 


Month 15 


6^A 


53.8 


16.7 


29.6 


3^.8 


38.6 


Month 22 


66.7 




32.6 


22.2 


25.2 


39.2 


Number In Sample: 














Baseline 


^3 


^2 


51 


27 


46 


209 


Month 6 


29 


27 


37 


18 


31 


142 


Month 15 




39 


46 


27 




202 


Month 22 


39 


38 


^^6 


27 




194 



Not in any paid or unpaid Job of at least * hours per week, or In a training or school program, 
''in some cases, baseline Interviews were administered several weeks after random assignment. 
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suggests that, without such programs as STETS, a high proportion of 
mentally retarded young adults are at risk of remaining inexperienced in 
activities designed to foster employability. 

Living Arrangement >^ Like many young adults, high proportions (78 
to 84 percent) of the control group members in the STETS sample lived with 
their parents, foster parents, or other adult relatives (see Table 
VII. 4). Only in St. Paul and Tucson were substantially higher-than-average 
proportions of controls living in other settings. Over time, the 
proportion of controls who were living with their parents tended to 
decrease only slightly in all sites. 

In general, the percentages who were living in supervised settings 

(group homes, supervised apartments, and institutions) remained fairly 

constant (about 10 percent), although these percentages ranged from 2 

percent or less in New York to over 26 to 30 percent in St. Paul. This 

variation across sites may reflect differences in the availability of 

supervised-living units, the differ^ nt methods whereby the sample members 

2 

were recruited from those living in such settings, and/or individual 
differences in the degree of independence. Modest increases occurred in 
the percentage of controls who lived independently (either alone or with 
non-family-related roommates), from 7 percent at baseline to 13 percent at 
the 22-month interview. This increase parallels the small movement away 
from living with parents. 

The STETS control group generally did not begin to establish their 
own families within the 22 months of observation. Only two persons lived 



1 

In addition to the living arrangements discussed in this section, 
we also considered the ^^revalance of incarceration among sample members, 
since this measure provides our only evidence on the impacts of the program 
on anti-social behavior. The incidence of incarceration was sufficiently 
low overall (one control at the 6-month interview and three at each of the 
15- and 22-month interviews) that no real conclusions can be drawn from 
these data. 

2 

For example, the St. Paul program recruited heavily (80 percent) 
from the state vocational rehabilitation agency (see Table 2.2 of Riccio, 
1984), which may have also arranged for placements in supervised 
residential settings. 
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TABLE niA 

LIVING ARRANGEMENTS OF CONTROL GROUP MEMBERS, BY SITE 



Type and 
Time Period 



Site 



Cincinnati Los Angeles New York St, Paul Tucson 



Total 



Percent Living with 
Parents or Foster Parents 

Baseline^ 
Month 6 
Month 15 
Month 22 

Percent Living In 
Supervised Setting^ 

Baseline^ 
Month 6 
Month 15 
Month 22 

Percent Living 
Independently^ 

Baseline^ 
Month 6 
Month 15 
Month 22 



90.9 


8^.1 


9^^.6 


66.7 


73.5 


83.6 


93.1 


7^^.1 


100.0 


72.2 


80.7 


85.9 


81.8 


79.5 


93. 


51.9 


70.5 


77.6 


87.2 


78.9 


89.1 


59.3 


68.2 


77.8 



9.1 


6.8 


1.8 


25.9 


10.6 


9.1 


6.9 


22.2 


0.0 


27.8 


6.5 


10.6 


6.8 


10.3 


2.1 


29.6 


^^A 


10.5 


7.7 


10.5 


2.2 


25.9 


6.8 


9.3 



0.0 
0.0 

11. 

5.1 



9.1 


3.6 


7.4 


16.2 


7.3 


3.7 


0.0 


0.0 


12.9 


3.5 


10.3 


*.3 


18.5 


18.2 


11.9 


10.5 


8.7 


14.8 


25.0 


12.9 


'f'f 


56 


27 


49 


220 


27 


37 


18 


31 


142 


39 


kl 


27 


44 


201 


38 


46 


27 


44 


194 


administered 


severcl weeks 


after 


random assignment. 





hkjmber In Samplet 
Baseline 
Month 6 
Month 15 
Month 22 



29 
44 
39 



This heading Includes group homes, supervised apartments, and Institutions. 

^Thls heading Includes living alone, living with unrelated rooir.i.ates, and living with spouse 
and/or own children (but not with other related adults). 
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with their own children at baseline, and none lived with a spouse; by month 
22, the number who were living with their own children had increased to 
seven, and a total of eight were living with a spouse. 



ESTIMATED PROGRAM IMPACTS ON ECONOMIC STATUS, INDEPENDENCE, AND 
LIFE-STYLE 

The analytic techniques that underlie our evaluation of the program 
impacts addressed in this chapter vary according to both the nature of the 
program outcome measures and the distribution of the sample in terms of the 
measures. For example, personal income, independence in financial 
management, the use of support services, and inactivity were analyzed with 
ordinary least squares (OLS) regression techniques. However, because 
living arrangement (a three-category variable in which approximately 80 
percent of the sample falls into one category) is not amenable to such 
techniques, we instead present comparisons of unadjusted percentages of 
this outcome. Similarly, family status variables have such skewed 
distributions that, with the relatively small sample sizes, tests of 
statistical significance would be highly unreliable regardless of the 
technique used. Thus, the results for these variables are discussed only 
briefly and for descriptive purposes. 

lo Program Impacts on Economic Status 

Table VII. 5 presents the program impact estimates on weekly 
personal income, along with the regression-adjusted mean values for both 
the experimental and control groups. In terms of in-program effects 
(i.e., the month 6 results), the results suggest that the STETS interven- 
tion significantly increased the total income of the experimental group by 
$21 per weeks Relatively little variation occurred in the estimated size 
of the income effects in this time period among the various sample sub- 
groups. The noteworthy exceptions are that larger-than-average estimated 



1 

Experimental -control differences in the percentage distributions 
of sample members by income level are presented in Appendix Table A. 14. 
These results indicate that ttic income gains associated with program 
participation accrued to individuals at all income levels. 
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T«t£ VII.5 

ESHK^TED PRCQRAM IffACTS (N lOlH. OOUJRS OF fitBVt 
PER30HM. INCOME, FOR TOTAL SmE )W) FOR KEY 
9JBGR01PS OF STETS PARHCIPAKTS 







Honth 6 






Month 15 




Month 22 




Subgroups Defined 
by Chtficterlstics 
at B«eline 


Experlnental 
QrouD Hean 


ODntrol 
(k'oup 
Nean 


EstlMted 
Inwt 


Experimental 
Ctcuo Nean 


Control 
Nev) 


EstlMted 
Inpact 


Experinental 
Group Nean 


Control 
Mevi 


Estimated 
Inpact 


Idt^ Salute 




50.9^^ 


20. 78** 


67.22 


59.67 


7.55 


71.59 


fi.39 


9.20 


SttB 

Clncinniti 

Nbw York 
St. P«il 
Tucson 


9.32 

07.70 

90.60 
63.65 
X).00 


^1.59 

69.01 
M.85 
«.18 


17.73 

26.09* 

21,79 

15.00 

20.82 


47.85 
69.47 
83.97 
63.61 
66.70 


39.13 
51.92 
88.27 
66.7^ 
51.11 


8.72 
17.55 

-3.13 
15.59 


49.64 
75.76 
88.96 
76.45 
65.85 


33.77 
59.66 
8^.59 
S9.47 
68.46 


16.07 
16.10 
4.37 
16.96 
-2.61 


Borderline 

HUd 

Moderate 


73.20 
69.56 
78.23 


^M3 
52.29 
58.63 


a9.07»» 
17,27** 
19,60 


65.37 
62.54 
104.68 


61.70 
56.21 
66.23 


3.67 
4.33 
38.65** 


75.59 
66.21 
69.95 


56.00 
67.31 
46.55 


17.59 
-1.10 
43.40** 


22 or oldBr 


68.69 


50.13 
53.15 


22. 64** 

15.74 


68.66 
63.85 


55.70 
68.76 


12.98* 
-4.91 


68.64 
77.61 


60.96 
65.57 


7.86 
12.24 


Qondrr 
FflMle 


66.07 


48.70 
53.88 


27.33»* 
12.19 


72.07 
60.69 


64.57 
53.07 


7.50 
7.fi 


80.41 
60.23 


63.46 
61.00 


16.95«^ 
-0.77 


Kne^ttntclty 
Black 
Hispv^lc 
White and other 


66.S2 
72.50 


42.33 
51. w 
55.74 


24.19»* 

31.81 

16.76* 


54.06 
87.63 
69.34 


52.47 
75.98 
59.41 


1.» 
11.65 
9.93 


63.34 
67.20 
72.06 


55.79 
64.85 
65.35 


7.55 
22.?5 
6.71 


t luTnn wIf'K i>Mi^j ■ 

Living in luperviaed 

setting 
Living independently 


65.97 


32. o7 
60.57 


16.82** 
5.40 


65.78 
62.42 


S9.64 
62.54 


5.94 
-0.12 


73.38 
55.94 


99.63 
69.55 


13.75** 
-13.61 


190.78 


18.19 


82.59** 


86.92 


54.68 


32.04 


72.50 


61.26 


-8.76 


Financial H»i yrt Skills 

Independent 
Not independent 


7^.57 
70.57 


^.89 
45.29 


9.68 
25.28»* 


88.51 
58.99 


63.76 
56.09 


24.75** 
0.90 


91.76 
63.90 


68.76 
59.95 


22.96* 
3.95 


Meoelpt of Tranalto 

Sa/SSDI 

Other transfers only 
No transfers 


100.06 
6d.fi 
50.26 


77.81 

45.as 

32.70 


22.25* 
23.36* 
17.56* 


90.69 
59.05 
51.06 


78.56 
54.18 
45.46 


12.13 
4.67 
5.60 


96.20 
69.86 
46.39 


72.80 
60.15 
54.04 


25.40** 
9.71 
-7.65 


CktSB or llBtardatlon 

Organl 


72.23 
71 o 

/I.W 


47.78 
CT 


24.45 
20.05** 


67.04 
67.25 


56.54 

6o.as 


10.50 
6.99 


78.37 
70.21 


67.94 

61 .as 


10.43 
6.95 


•ortc GcpMiwBi In Jm 
Ymts Prior to EhrolJMt 
Regular job lasting 

>3 K>nths 
OtKer >>b lasting 

>3 Months 
Otivr 


77.99 
^.88 
67.36 


55.34 
53.09 
47.95 


22.65 

21.79** 

19.43** 


87.63 
68.46 
60.58 


79.62 
56.89 
54.47 


6.01 
9.57 
6.11 


dE».64 
71.99 
67.50 


8^.45 
62.12 
56.22 


0.39 
9.67 
11.28 


School Staftjs «t Refaml 

Enrolled 
Not enrolled 


66.63 
73.96 


42.49 
54.66 


24.14* 
19.30** 


59.67 
70.56 


58.56 
60.16 


1.11 
10.40 


66.86 
73.76 


59.19 
63.86 


7.67 
9.90 


Njiter in Sample 






274 






391 






392 



NOTEt Th«ise reailts were estUutod throus^^ ordinary least squares techniques. In additkn to tt» control varlAles that «e Included in tfm 

■cdel which underlies the overall net imfoct estlMates reported for the total saaple (see Appendix Tible A.I), the nodel include wiAles 
that interacted the cxperlnenfcal status vartable with the subgroup v.iriibles. The nuwber of cases in each aub^txjp is reported in 
Appendix Table A. 12 



•Statistic:ally significant at the 10 percent level, two-taUed test. 
**Statisticjally signifioant at the 5 percent level, two-tailed test. 
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effects occurred for males, blacks and Hispanics, and individuals under the 
age of 22 — groups that have traditionally exhibited high youth unemployment 
rates, even without intellectual or other disabilities. Other groups which 
experienced large in-program effects were those who were living 
independently at baseline and those whose IQ scores indicate borderline 
retardation. 

For the postprogram period, a different pattern of results is 
observed. Estimated impacts on personal income are no longer large or 
statistically significant for the total sample; the point estimate of the 
overall effect in month 22 is $9.20 per week. The only subgroups which 
showed sustained evidence of income gains at months 15 and 22 were those 
with moderate retardation, males, Hispanics, those who were living with 
parents, those who exhibited independence in financial management skills at 
baseline, and those who received SSI/SSDI at referral. With the exception 
of the SSI/SSDI recipients and the moderately retarded individuals, each of 
these subgroups experienced earnings gains that roughly equalled these 
total income effects (see Appendix Table A.4). For the moderately retarded 
and the SSI/SSDI recipients, the impacts are attributable to sizeable 
effects on both earnings and SSI/SSDI (see Table VI. 4 in Chapter VI and 
Appendix Table A.4). 

2, Program Impacts on Independence and Life-Style 

The estimated impacts of STETS on independence in financial manage- 
ment and the use of social support are generally positive but small (see 
Table VII. 6). By month 22, virtually no difference between the experimen- 
tal and control groups existed in terms of independence In financial 
management, the receipt of formal services, or personal relationships with 
benefactors. The only noteworthy impact is the increased percentage who 
used formal services (from 79 to 88 percent) during the in-program period 
(month 6), which is likely to be a direct result of participating in STETS. 

Similarly, STETS had no impact on whether sample mt^rabers were 
inactive, except during the period of program participation itself. As we 
indicated in Chapters IV and V, a number of offsetting impacts on 
activities occurred in the postprogram period. While the experimental 
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VII.6 



esnmTED PROGRAH WhOS CH SOCIAL SIPPCRT, ItCEPEM»«E, KD imcHVITY 







Month 6 






Month 15 




MDnth 22 




OutcoBe noasuroo 


Experlflental 


Control 


Estlmted 

I-^Pfict 


Experimental 
Creyp Hea^ 


Oon(rol 


EstiAated 


ExperlMental 


Oxtrol 


Estimated 

L-Rpact 


Peroert Who Oenomtrate 
Independence In Fln«)clal 


36*9 


3.1 


7*6 


37*5 


33.2 


2,3 


36.3 


35.5 


0.8 


Peraeft ReoelvlnQ Any 
Seryloet Fro« Agency^ 


87.9 


78.7 


9.2»» 


70. 5 


68.1 


2.^ 


68.2 


65.3 


2.9 


Peroert With Benefactor^ 


31.6 


30.2 


1.^ 


31.1 


22.6 


8.5* 


2<^.7 


19.8 


^9 


Percent Inactive^ 


20.1 


36^ 


-16.1»» 


«.3 


38.2 


4.1 


*1.3 


39.7 


1.6 


HjMber In Sa^>le 






288 






^16 






40^ 



NOTE: These rMults were ettlMted through ordinary least squares techniques. Definitions and aeans of control varl^les that are Included In 
the aodels are presented In Appendix Table A*1. 

a 

Independence U defined as performing without assistance at least two financial management activities (shopping, handling fallls» and using bank 
accounts), and without assistance In the other financial management activity. 

b 

The services that were provided by agency staff meidljers included job training, >>b»search asslstanoe, schooling, residential counseling, other 
counseling, financial management (assistance with shopping, bUl paying, or handling bank accounts, or receiving transfer-program benefits on 
behalf of the Individuals), and transportation (providing regular transportation to >ob, training program, or school). 

c 

Benefactors are individuals named by primary sample members as prcv^ ng assistance In two or more of the following areast Job search, 
residential counseling, other ounellrg, financial management, and transportation. These individuals could be relatives or friends of the 
aasple member, or service agency staff menbers. 

d 

Not In any paid or unpaid job of at least 4 hours per week, or In a training or school pros^am. 

♦Statistically si^Dif leant at the 10 percent level, two-tailed test. 
**Statlstlcally significant at the 5 percent level, tso-tailed test. 
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group was more likely than controls to hold a regular job, fewer were 
enrolled in training or schooling programs. Once ST2TS participation was 
over, experimental group members were no more likely to be involved in some 
form of employment, training, or educational activity than they would have 
been had they not participated in STETS. 

3« Impacts on Living Arrangement 

For three reasons, we assess the impacts of STETS on living 
arrangement by correlating percentages in various living arrangements at 
baseli ne wi th thos e at each of the f o llow-up periods : ( 1 ) livi ng 
arrangement is a three-category outcome measure; (2) most of the sample 
members lived with their parents at baseline; and (3) experimental -control 
differences in living arrangements existed at baseline. As shown in Table 
VII. 7, these tabulations show no evidence that STETS had a strong or 
consistent impact on the proportion of the sample who were living with 
their parents or with other adult relatives during the in-program period; 
in month 6, most of chose who had been living at home at baseline continued 
to do so, and approximately equal proportions from both the experimental 
and control groups who had been living elsewhere had moved in with their 
parents by month 6. There is evidence that STETS descreased the likelihood 
of living in supervised settings during this in-program period, in that 
more experimentals than controls moved out of such arrangements, and more 
control group members than experimentals entered them. An estimated 
positive in-program impact of STETS on living independently was due to a 
different pattern of movements— more of the experimental group who had been 
independent at baseline remained independent, and more moved from other 
arrangements into independent living arrangements. 

The postprogram results show that most of the sample continued to 
live in the same arrangements as at baseline. This finding was especially 
true for those who were living with their parents, but less so for those 
who were living in supervised settings. STETS did have some impact on the 
pattern of movements that occurred; the most substantial of these impacts 
occurred by month 15. At month 15, experimental group members were more 
likely than those in the control group to leave independent living arrange- 
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IPOLE V1I.7 



eSTlH^TED PfWtaWM I^f AaS ON LIVING AflRANCDCW, 
BY LIVING ARfWCE>CKTS AT B^SELPC 
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Month ZZ 
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Expert- 
Mental 

C^oup He«i 


Oontrol 
Qroup 


Estimted 
Iii«>act 


Expert- Oontrol 
Mental (^oup 
Ooup Mean Mean 


Estimated 
iMpact 


Expert- 
Mental 

Group Mean 


Control 
Croup 
Mean 


Estimated 
iMpact 



Living with ParenU 97.^ 95.2 2.2 88.1 89.5 -1.^ 88.3 88.^ -0.1 

Living In Supexvlsod Setting^ 11.8 Y).0 1.8 13.0 5.9 7J JO.^ 16.7 13.7 

Uvlng IndependbflUy^ 13.3 16.7 -3.^ 27.3 9J 16*2 21.7 27.3 -5.6 



»Uter In Swple 236 313 307 
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Month 15 
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Living r rranoenent at Baseline 


Experi- 
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Qroup Mean 


Cdntrol 
Qtomp 
Mem 


Estimated 

L^MCt 


Experi- 
Mental 
(^oup Mear 


Control 
Qrai|} 
Mean 


Estimated 
Lvact 


Experi- 
Mental 
<^aufi Mean 


Control 
Qroup 


Estlmited 


Living with Parents 

Living in Supervised Setting 

Living Independently^ 


0.9 
70.6 
0.0 


^0 
90.0 
16.7 


-3.1 
-19.* 
-16.7 


5.9 
65*2 
13.6 


4.1 

76.5 
0.0 


1.8 
-11.3 
13.6 


5.5 
43.5 
8.7 


4,3 
61.1 
0.0 


1.2 
-17.6 
8.7 
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mental 
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QrojD Mean 


Mean 
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QroLtp Mean 


Mean 


Inpact 


Living with Parents 


1.7 


o.a 


0.9 


5.9 


6.4 


"0.5 


6.1 


7.3 


-1.2 


Living in Supexvlsed Setting^ 


17.7 


0.0 


17.7 


21.7 


17.7 


4.0 


26.1 


22*2 


3.9 


Uving Independerfcly^ 


86.7 


66.7 


2D.0 


».1 


9D.9 


-31.8 


69.6 


72.7 


-3.1 


Nu^wr In Sample 






23 






S2 






56 



NOTEt These results act based on cross-Ubulationa of living arrangeMent at tsaseline with those at each follo»-<jp observation period. Sanple 
sizes were too small to permit cM-square tests of statistical slgnlfloance. 

a 

This heading Includes group homes, supervised apartments* and Institutions, 
b 

This heading Includes living alone, living with inrelated rooMWtes, and living with spouse and/or om chllckren (but not with other related 
adults). 
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ments, either to return home or to live in a supervised setting; by month 
22, the experimental and control groups exhibited few differences in terms 
of the percentages who were making these types of moves. More experimental 
than control group members moved away from supervised settings at both 
postprogram observations, and experimentals were more likely than controls 
to move from these settings into the homes of their parents. 

The program had no apparent effects on family formation. By month 
22, nine experimentals and eight controls were living with a spouse, and 
six experimentals and seven controls were living with their own children. 

C. SUMMARY AND CONCLUSIONS 

Our discussion of the impacts of the STETS program in previous 
chapters focused on those areas of independence that pertain directly to 
employment. We found that participation in a transitional-employment 
program increased employment in the competitive sector, which in itself is 
a move toward a more independent life-style. It also increased earnings, 
while somewhat decreasing dependence on cash transfer programs. 

In this chapter, we observed some relatively small program effects 
on such measures of independence as overall economic status, services 
received from community agencies, and involvement in activities oriented 
toward employment. However, these effects generally declined to a great 
extent in the postprogram period, seemingly due to two factors. First, in 
the later observation periods, either the direct effects of STETS 
participation on such outcomes as total income, service utilization, and 
level of inactivity either were no longer evident or, where they were 
evident (as with personal income), the estimated effects were not 
statistically significant. Second, while the STETS experience provided a 
head start toward independence for many sample members, those who did not 
participate in the program also began to achieve similar levels soon 
afterwards. Although certain subgroups (for example, Hispanics and those 
with a moderate level of retardation) did seem to continue to benefit from 
the program, at least in terms of personal income, even those who were more 
likely to achieve and maintain positive effects from their experience in 
STETS exhibited relatively low levels of independence by the end of the 
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observation period: most of the mentally retarded young adults in the 
study had total personal incomes that were less than the poverty level, 
thereby restricting their opportunities for achieving a more economically 
self-sufficient life-style* Possibly because of their low incomes, most 
continued to live with their parents and to exercise little independence in 
financial management. Finally, a substantial proportion of both the 
experimental and control groups were not involved in employment, school, or 
training, and thus had limited opportunities to gain valuable work 
experience and skills. 

Given the short postprograra period for which we have data, we 
cannot tell whether a more economically and socially independent life-style 
would eventually be achieved by these mentally retarded young adults, or 
whether the effects of participating in a transitional-employment program 
would become more evident at a later period. 
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VIII. BENEFITS AND COSTS OF STETS 



As we mentioned In Chapter I, a key policy issue behind the STETS 
demonstration was whether the impacts of the program would be sufficiently 
large to justify its costs. In this chapter, we examine this issue by 
comparing the impacts discussed in the previous chapters with estimates of 
the program costs. This analysis provides a framework for assessing the 
success of STETS and for interpreting the various impact estimates. 

Benefit-cost analysis uses dollar values as a common denominator to 
draw comparisons between the diverse program impacts and costs. This 
analysis assigns dollar values to the impacts, suras the resulting values, 
and compares them with the cost estimates. In doing so, it inevitably 
excludes some impacts, either because evaluation resources were in- 
sufficient to enable us to measure them or because the impacts are 
essentially intangible. In other cases, impacts were measured but were 
difficult to appraise, and value precisely. To deal with these problems, we 
used a comprehensive accounting framework that lists all the major benefits 
and costs regardless of whether we measured them. This framework facili- 
tates an analysis of the extent to which we have measured and included the 
important benefits and costs, and whether reasonable conclusions can be 
djrawn on the basis of these measured Items. We also made several 
alternative estimates of the value of impacts that were measured; each 
alternative was based on alternative assumptions about the appropriate 
value that should be assigned to specific impacts. This set of plausible 
alternatives provides a far better picture of the; overall success of STETS 
than would a single. Inherently imprecise, baneflt-cost comparison. 

The result of this analysis Is a qualified conclusion that STETS 
represents a good social Investment. If operational costs can be held at 
the level observed when the demonstration was operating relatively smoothly 
(I.e., during the five-month "steady-state" period), then the benefits 
generated by STETS can be assumed to outweigh its costs, as long as the 22- 
raonth Impacts on experlmentals persist for at lectsC seven months beyond the 
observation period. These benefits accrue primarily to program partlcl- 
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pants, who gain from their increased earnings (both while in STETS and in 
subsequent employment). Although the rest of society also receives 
substantial benefits, it must pay the taxes that fund the STETS services. 
Consequently, they incur a net financial cost from STETS in the short-run, 
although this cost may be offset by savings from long-run reductions in the 
use of sheltered workshops by participants, and by the desirability of 
increasing the opportunities available to and enhancing the social 
integration of mentally retarded young adults. 

We begin our analysis by providing an overview of the benefit-cost 
accounting framework and the general analytic methods. We then present the 
estimated costs of the program, and discuss the value of changes in (1) 
the output produced by experimentals both in and out oi: STETS, (2) their 
use of sheltered workshops, school and training programs, <sad residential 
programs, and (3) their dependence on public financial assistance. We 
aggregate these individual measured costs and benefits in the framework and 
analyze them and their relation to unmeasured effects in order to assess 
STETS overall as an investment. A separate technical appendix (Appendix C) 
provides additional details on the estimates, procedures, and findings of 
this benefit-cost analysis. 

A. BENEFIT-COST ANALYSIS FRAMEWORK AND METHODS 

While countless questions can be asked about a program such as 
STETS, benefit-cost analysis is appropriate for addressing only two: Was 
the program economically efficient? And was it equitable? Efficiency 
pertains to the effect of a program on the total value of the goods and 
services available to society (i.e., was the value of the goods and 
services available to society greater because of STETS, or would their 
value have been greater if the resources that were devoted to STETS wore 
used for alternative purposes?). Equity pertains to the distribution of 
goods and services among groups in society, how STETS affects that 
distribution, and ether a specific group of individuals benefits or 
loses. 

In general, the methods used in benefit-cost analysis are oriented 
toward addressing efficiency: program efficiency can usually be de- 
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termined, and it is presumed that greater program efficiency is 
desirable. Equity questions a.-e more difficult to answer. Benefit-cost 
analysis can (to some extent) determine the effect of a program on the 
distribution of resources, but contains no special criteria for judging 
whether a distributional change is desirable. Thus, addressing equity 
entails a more descriptive analysis, and conclusions must be based on a 
broader analysis of public policy and social concerns. 

The basic method used to determine economic efficiency is to assign 
dollar values to all estimated impacts and costs. These values are then 
summed together to yield an estimate of the program's net present value- 
that is, the difference between benefits and costs, where all dollar values 
are adjusted to reflect their value in a specific base period. A positive 
net present value indicates that the resources are being used ef- 
ficiently. 

While the net present value criterion can easily be stated, a high 
degree of uncertainty surrounds its estimation: program effects are 
measured imperfectly, and some cannot be estimated at all; uncertainties 
surround the values that should be placed on the specific program effects 
or costs; and the appropriate techniques necessary to aggregate individual 
benefits and costs inherently involve numerous approximations. Conse- 
quently, it is difficult to apply the net present value criterion to judge 
the economic efficiency of the program. 

Because of the error associated with any single estimate of net 
present value, much of the usefulness of benefit-cost analysis pertains to 
its comprehensiveness. The process of drawing together measures of the 
various inputs and outcomes and the general patterns that emerge from the 
attempts to assign relative values are often more useful than any specific 
estimate of net present value. For this reason, the analysis does not 
focus on a single net present value estimate but, instead, on a set of 
estimates. By examining the different assumptions, the underlying outcome 
estimates, and the techniques used to value outcomes, it is possible to 
identify those aspects of STETS and its evaluation that are most important 
in determining the overall findings— that is, which (if any) aspects, if 
changed, would change the basic nature of the findings. 
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The core of this approach is a benefit-cost accounting framework 
that imposes a logical rigor on the anal>sis and serves as a guide for 
interpreting the results* ThG framework specifies a consistent method for 
valuing the diverse sets of effects* The approach used here is based on 
concepts similar to those that underlie the estimation of gross national 
product (GNP). It focuses on the net resource gain or loss induced by 
STETS as it was implemented in the demonstration. Essentially, the 
approach entails estimating the change in resources available because of 
STETS, and then valuing those resources at their market cost. Thus, for 
example, STETS-induced reductions in the use of sheltered workshops by 
experimentals enables society to reallocate some of the resources that 
would have been used to operate those programs. The market value of the 
saved resources is used as a measure of the value generated by the 
reduction in sheltered workshop use. In general, this valuation procedure 
is convenient to use and does not necessitate attempting to measure such 
difficult concepts as the willingness of society to pay for the various 
outcomes. 

While this procedure assigns a value to program effects, that value 
will be viewed differently by different groups. For example, experimentals 
who lose SSI benefits after obtaining a job will view the loss as a cost, 
while taxpayers will view it as a financial gain. The ac^^ounting framework 

captures these differences through three analytical perspectives: society 

2 

as a whole, program participants, and nonparticipants. The perspective of 
society as a whole abstracts from all of the redistributional aspects of 
STETS and focuses on its efficiency, since it considers only the use of 



1 

Of course, one possible reallocation would be to continue to 
operate the sheltered workshops at the same scale and to serve clients who 
would not have been served in the absence of STETS. 

2 

The nonparticipant group includes everyone in society who is not 
given the opportunity to participate in STETS. Thus, it encompasses much 
more than the control group, which comprises a very small part of the 
nonparticipant group. We prefer this terra to the more common term 
''taxpayer group," because the participant group also pays taxes, and 
because not all of the effects on nonparticipants occur through the tax 
system. 
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resources. Transfers of income between groups are assumed to cancel each 

other out in the social perspective—that is, a dollar of benefit or cost 

to one personals assumed to be equivalent to a dollar of benefit or cost to 

anyone else. The perspectives of participants and nonparticipants 

facilitate an analysis of the distributional consequences. For each group, 

the question for net present value is the same as it was for society as a 

whole: Does net present value (from that perspective) exceed zero? Do 

participants gain or lose, on average, from their participation? Are their 

earnings gains and increased independence sufficient to outweigh the losses 

of transfer benefits such as SSI? How are nonparticipants affected? Does 

2 

STETS require a net subsidy from nonparticipant taxpayers? 

All three perspectives mask differential effects on specific 
individuals or groups. The impacts of STETS are measured as averages and 
indicate the expected effect of STETS. Obviously, participants will differ 
in their response to STETS and may do better or worse than the statistical 
averages. In addition, individual nonparticipants will be affected 
differently. The employer who is able to hire a productive worker bec^^tse 
of STETS will perceive the program differently than will an average 
taxpayer who helps to fund STETS. Therefore, the STETS benefit-cost 
estimates must be taken as indicative of the expected overall effects of 
STETS, viewed from a broad perspective that is appropriate for judging 
aggregate program performance. A more detailed analysis would be required 



1 

Of course, any resources consumed in transferring income would be 
counted as costs from the social perspective. 
2 

One analytically useful feature of using these three perspectives 
is that the sum of the net present values calculated from the participant 
and nonparticipant perspectives equals the net present value for the social 
perspective. This "adding-nip" property is valid because participants and 
nonparticipants constitute mutually exclusive groups that, when combined, 
include all members of society. Therefore, transfers of income between 
these two groups cancel each other out in the social perspective, because 
the benefit to one group is assumed to be equal to the cost to the other. 
Benefits or costs that accrue to one group and that are not offset by 
corresponding costs or benefits to the other (e.g., increased work output) 
do not cancel out when added, and they thus represent net social benefits 
or costs. 
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to answer questions about wh^ither specific types of individuals would 
benefit from enrolling in STETS# 

Because the benefits and costs of STETS occur over time, the 
analysis must compare streams of benefits with streams of costs« To 
simplify this task, we include in the accounting framework several pro- 
cedures for aggregating dollars in different time periods and for producing 
equivalent estimates of benefits and costs at a single point in time. To 
do so, it is necessary to account for differences in the value of benefits 
and costs across time periods due to inflation and to foregone interest 
earnings^ The inflation differences are corrected by valuing all benefits 
and costs in 1982 dollars. Thus, differences between benefits and costs 
reflect real changes in resources, not changes in the nominal value of a 
dollar. The differences due to foregor- interest reflect the fact that a 
benefit that occurs in the future is worth less than the same benefit that 

occurs today, because today's savings could be invested and would earn 

2 

interest in the future. The procedure for adjusting for such differences 
is called "discounting," and its importance is well established among the 
analytic literature (see, for example, Gramlich, 1981). The only 
uncertainty that remains pertains to the interest rate that should be used 
in discounting future benefits and costs. We assume a 5 percent real 
annual rate (that is, a rate calculated by netting out inflation) for our 
benchmark, and test the importance of this assumption by calculating 
alternative estimates using real annual rates of 3 and 10 percent. 



1 

The subgroup analyses in Chapters IV through VII provide some 
information on the impacts of the program on individual types of 
participants. 

2 

For example, suppose that a ^11,000 benefit occurs 10 years from 
now. The issue would then be, what present value invested today would 
yield $1,000 ten years from now? If interest rates were, for example. 5 
percent, then this present value would equal $1,000 divided by (1 + .05)^^, 
or $614. Gramlich (1981) describes this process in more detail. 
3 

The Office of Management and Budget (1972) mandates a 10 percent 
discount rate for evaluating government programs. 
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(Appendix C provides further information on the rationale and procedures 
for discounting.) 

Table VIII. 1 presents the benefit-cost accounting framework. The 
table lists the major impact components of STETS (regardless of whether we 
can value them) and suggests whether a component is, on average, a benefit, 
a cost, or neither from each of the three perspectives.^ The table also 
indicates what data sources are used to measure and value the effects or 
Whether a particular effect is left unmeasured. The next two sections 
discuss the separate cost and benefit components. 

Before proceeding, however, it is important that we review how the 
impact estimates presented in Chapters IV to VII are used in the benefit- 
cost analysis. These estimates indicate the effect of STETS on experi- 
mentals at 6, 15, and 22 months after randomization. These "point-in-time" 
estimates are adequate measures of the impacts of STETS, but are inadequate 
for the benefit-cost analysis, which requires information on the impacts of 
the program for the entire 22 months. In order to compare benefits and 
costs, we need to estimate the cumulative change in earnings, program use, 
transfer receipt, and other activities. In the absence of continuous dati 
on these activities, it is necessary to derive cumulative measures by 
interpolating between the point-in-time estimates. 

Any interpolation method involves some arbitrariness. We have 
chosen to interpolate linearly between the point estimates. This method is 
straightforward, and appears reasonable in that no alternative is clearly 
preferable. Thus, although all program impacts used in the benefit-cost 
analysis are inherently more imprecise than the specific impact estimates 
presented earlier, we feel that estimates of cumulative effects based on 
linear interpolations provide an accurate indication of the true magnitude 



1 

Whether an impact component will be a net benefit or cost is 
sometimes problematic. Table VIII. 1 reflects prior judgments about the 
value of components from the three perspectives. The treatment of all 
components in the final net present value calculations is of course 
determined by the estimated actual effects of STETS. 
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TABLE Vin.l 

EXPECTED BENEFITS AND COSTS OF STETS BY ANALYTICAL PERSPECTIVE 









Analytical Pers 


pective 


a 


Component 


Social 


Participant 


Nonparticipant 


Data Source 


I. 


Prograa Costs 












Project operations 


— 


0 


— 


A 




Paynents to participants 


0 


+ 




A 




Central administrstion 


- 


0 


— 


P 


II. 


Outfrat Produced by PsrtlcipsnCs 












Phsse 1 and Phase 2 output 


+ 


0 


+ 


S 




Oiitout forffone while In STETS 


— 


— 


0 






Increased out-of-program output 


+ 


+ 


0 




III. 


Other PrograM 












Reduced use of: 












Sheltered workshops 


+ 


0 


+ 






Work-activity centers 


+ 


0 


+ 






School 


+ 


0 


+ 






job^trainlng programs 


+ 


0 


+ 






Case-management services 


+ 


0 


+ 


b 




Counseling services 


+ 


0 


+ 


E 




Social/recreational services 


+ 


0 


+ 


E 




Transportation services 


+ 


0 


+ 


E 


TW 
JlV. 


EsesMieiiCMX oi.diaui.oii 












Reduced use of: 












Institutions 


+ 


0 


+ 






Group homes 


+ 


0 


+ 


I,P 




Foster homes 


+ 


0 


+ 






Ociu xnucpvnuciik tes iuciii>i<tx 


+ 


0 


+ 


I»P 




programs 










?. 


TraoBfer Payaects and Taxes 












Reduced SSI/SSDI 


0 


— 


+ 


I»P 




Reduced other welfare 


0 


— 


+ 






Reduced Medicaid/Medicare 


0 


— 


+ 






Increased taxes 


0 


— 


+ 


I»P 


VI. 


Transfer Adnlnlstratlon 












Reduced use of SSI/SSDI 


+ 


0 


+ 


I»P 




Reduced use of other welfare 


+ 


0 




T P 




Reduced use of Medicaid/Medicare 


+ 


0 


+ 


I»P 


VII. 


Intaqglbles 








u 




Preferences for work 


+ 


+ 


+ 




Increased self-sufficiency 


+ 


+ 


+ 


u 




Increased variation in 








u 




participant income 








u 




Foregone nonmarket activity 






0 




Increased independent living 


+ 


+ 


+ 


u 



NOTE: The individual components are characterized from the three perspectives as being a net benefit 
(+), a net cost (-), or neither (0). 

*The codes used for the data sources are as follows: S-special study, I-interview data, P-published 
data source, A-STETS accounting system data, U-item not measured, and E-item measured but excluded 
because the effects of STETS were trivial. 
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1 

of program impacts. (Details on the interpolation procedures are provided 
in Appendix C.) 

B. STETS PROGRAM COSTS 

The accounting framework disaggregates costs into three 
components: the operating costs of the projects, compensation paid to 
clients while they were in Phase 1 or Phase 2 activities, and central 
administrative costs. The operating and ceatral administrative costs are 
paid by nonparticipants. Because these costs represent the value of the 
resources used to operate STETS, they also represent social costs. 
Participant compensation is treated as a transfer from nonparticipants to 
participants, because it represents a shift in resources from one group to 
another. 

1. Operating Costs 

During the 27 months of operations, the five projects served 284 
clients anc ported operating costs of almost $2,500,000, implying average 
operating costs of $8,800 per client. However, for two reasons, this 
estimate is misleading for the benef it--cost analysis. First, it 
corresponds to all clients, and not to the group of 226 participants whu 
were included in the research sample. Second, it includes costs that are 
attributable to the fact that STETS was a demonstration. 

The first problem can be corrected in part by adjusting for 
differences between experimentals and other clients in their length of 
program participation. We estimated the cost of serving an active client 



1 

Data on program costs and participation in STETS were obtained 
from demonstration accounting and client-tracking records. Because these 
records provided data for the full observation period, interpolation 
between point estimates was unnecessary for estimating the following: 
STETS program costs, STETS payments to participants, and the value of 
output produced by participants while in Phase i and Phase 2 jobs. 
2 

The output produced in Phase 1 and Phase 2, which is related to 
this compensation, represents real resource gains. Section C discusses 
both this output and changes in output produced outside of STETS. 
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for a month and then raultiplied this average cost by the observed average 
length of time during which experiraentals were active in the 
demonstration. This method is accurate as long as the five demonstration 
projects provided the same level of service per month to experimentals as 
they did to clients who were not in the research sample. Because there is 
no evidence of such differences in service provision, we feel that the 
method is sound. 

It is more difficult to correct for the special demonstration 
costs. Our general rule was to include all costs of the demonstration as 
it was fielded, with two exceptions. First, we subtracted an estimate of 
research-related costs — the costs of finding and screening applicants who 
were ultimately assigned to the control group, of completing and processing 
the research data collection forms, and of staff time spent with the 
researchers. Riccio and Price (1984) estimated that these costs 
constituted 5 percent of total project expenditures (including both 
operating and participant compensation expenditures). Because these 
research costs probably had little or no effect on the impacts of the 
program, we feel that they should be excluded from the benefit-cost 
analysis. 

We made the second exception to the principle of estimating 
operating costs as actually incurred in the demonstration because several 
demonstration-specific features made the observed costs abnormally high. 
In particular, the limited duration of the demonstration meant that the 
actual costs overrepresented the higher average costs of the initial 
project implementation and of the demonstration phase-down period. 
Addi tional cos ts were also incurred because projects found it necessary to 
take special precautions to deal with the funding uncertainties surrounding 
the demonstration itself. Riccio and Price (1984) discuss these problems 
and suggest that the costs incurred during a five-month "steady-state" 
period best represent the costs of operating STETS on an ongoing basis. 
This period (which covers slightly different months at each site) fell in 
mid- to late-1982, a period when enrollments were high and operations were 
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stable relative to earlier and later periods. We have used average costs 
from these periods in the benefit-cost analysis. 

After making these two adjustments for research costs and the 
effects of start-up and budgetary uncertainties, we estimate that it costs 
$666 per month of active participation to provide STETS services.^ 
According to data collected by MDRC as part of its client-tracking system, 
experimentals were active for an average of 9.3 months. When discounted at 
a 5 percent real annual rate to the point of enrollment in STETS, the 
implied participation cost is $6,050 per participant. 

2. Participant Compensation 

Participant compensation includes the wages and fringe benefits 
that the five projects paid to participants in Phase 1 and Phase 2 
activities. It also includes wages paid by employers directly to 
participants in Phase 1 and Phase 2. The projects reported all these 
expenditures monthly to MDRC as part of the demonstration monitoring and 
accounting system. During the five months of the steady-state period, 
participant compensation averaged $341 per month of active participation 
(average compensation expenditures over the entire demonstration were $30 
lower than during the steady-state period). Given the estimated average 
length of active participation (9.3 months), this figure implies that 



1 

As mentioned in Chapter IV, employment impacts were greatest for 
persons who were served during the steady-state period, suggesting that the 
higher average costs during the demonstration start-up and phase-down 
periods occurred because the projects were establishing new procedures and 
dealing with inefficiencies due to smiiil and changing scales, rather than 
because extra services were being provided. This finding further supports 
the use of steady-state costs in the benefit-cost analysis. 
2 

Riccio and Price (1984) reported costs on the basis of enrollment 
months rather than active months. Because some clients were inactive for 
part of their enrollment period, the cost per enrollment month will be 
lower than the cost per active month. However, the average length of 
participation will be correspondingly higher when measured in enrollment 
time rather than in active time. Thus, the cost per client (the product of 
the cost per month and the months of participation) is independent of how 
time is measured. (Appendix C discusses the cost-estimating procedures in 
greater detail.) 
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participants received Phase 1 and Phase 2 compensation worth $3,094 per 
participant when discounted to the time of enrollment. 

3. Central Administrative Costs 

Central administrative costs cover the activities necessary to 
administer the contracts with the five projects and to provide 
demonstration-wide coordination. MDRC performed this task in the 
demonstration, although state or federal agencies would probably assume 
this role in an ongoing program. For example, a state vocational 
rehabilitation agency could fund the programs and would assume 
responsibility for audit and performance monitoring. Estimating central 
administrative costs was difficult in STETS because of the overlap between 
MDRC*s monitoring and research activities. Their dual role in the 
demonstration meant that their costs exceeded the central administrative 
costs that would be incurred in an on-going program. In addition, they 
incurred substantial start-up costs as they selected the sites acd helped 
them operationalize the program. Consequently, the demonstration 
experience does not provide an adequate guide to estimating future central 
administrative costs. Our benchmark estimate is that central 

adioinistrative costs would be approximately $20 per month if STETS were 
operated in a fairly decentralized manner whereby most of the monitoring 
would focus on audit responsibilities and fairly straightforward 
performance measures. Costs would be higher if the central authority 
provided intensive monitoring or technical assistance. 

C. OUTPUT PRODUCED BY PARTICIPANTS 

The analysis of STETS-induced effects on output produced by 
participants distinguishes between goods and services produced by 
participants in Phase 1 and Phase 2 and those produced by them outside of 
STETS. These two types of output have different distributional 
consequences and necessitate using different estimation techniques. 
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!• Value of In-Program Output 

The value of output produced by participants in Phase 1 and Phase 2 
is an important program benefit. This output accrues to nonparticipants 
(and to ^society) and has been very important in previous benefit-cost 
studies. One measure of the value of in-program output is the revenue 
generated either by payments made by firms which used participant labor or 
from the sale of participant-produced goods. This valuation method 
provides a reasonable lower-bound estimate of the value of in-program 
output, because the output was actually purchased for this amount. 
However t the STETS sites did not pursue revenue-generating strategies as 
their primary goal; rather, they focused on securing placement and training 
for the STETS participants. Thus, revenue may seriously underestimate the 
actual value of output. Based on estimates by Riccio and Price (1984), 
revenue for the five-month steady-state period was over $108,000, or $131 
per active participant month. Thus, revenue offset almost 40 percent of 
participant compensation. 

As an alternative method for estimating the value of the in-program 

output of participants, we conducted a series of work-activity case studies 

3 

for 33 randomly selected experimentals* For each person, we estimated the 
net value of the output they produced during a two-week reference 
period. This estimate was based on the alternative supplier's price of 



1 

For example, in the national Supported Work demonstration, the 
value of output offset approximately 65 percent of the social costs (see 
Kemper and Long, 1981, p. 269). 
2 

Reveme is a lower-bound estimate under the assumption that 
profit-seeking employers would not pay more than a product (participant 
labor service) was worth to them. Thus, the value of output should not be 
less than what was actually paid. Of course, this argument is weakened if 
altruism prompts employers to overpay because of their desire to support 
the STETS program and its participants* In either event, the direct 
estimates of the value of output (discussed in the text) offer a more 
accurate estimate of the resource value of parcicipant output. 
3 

We also studied seven participants who were not included in the 
research sample. The results for these participants were similar to the 
results for the 33 experimentals (see Appendix C). 
4 

The studies were completed between September and December 1982, 
the period that is generally considered to be part of the "steady-state" 
period. 
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the participant output— that is, on the wages and fringe benefits that 
would have been paid by an employer to other workers to produce the output 
that was produced by the participant. This estimate assumes that employers 
can obtain additional labor at the wages paid to their regular employees. 
To obtain the net value added by participants, we subtracted from the 
alternative supplier's price an estimate of the costs of additional 
employer-provided supervision and reduced output from other workers from 
using the participant labor. 

These estimates of the net output are based on in-person interviews 
with the participants' supervisors, and rely on those supervisors' 
judgments about participant productivity, supervisory costs, the reactions 
of other staff, and the source and cost of alternative labor to perform the 
participants' tasks. These estimates are also subject to problems because 
of the difficulty in assessing whether the use of STETS-p^rticlpant labor 
enabled the employer to increase output. There were several instances 
where an employer took on a STETS participant and raised the quality of the 
output, rather than increasing the amount or price of that output. For 
example, one STETS participant was hired by a day-care program operator. 
The operator did not change prices or increase enrollments, but, instead, 
apparently used the STETS participant to increase the amount of attention 
given to the children in the day-care program. Our net value-of -output 
estimates do not adjust for such quality changes. Thus, our estimates may 
be too low in some cases in which quality changes occurred. 

Despite these limitations, the estimates of the net value of 
participant output represent useful indicies of the value of in-program 
output. They are based on careful, systematic case studies, and, where 
possible, incorporate actual wage, fringe-benefit, and production records 
in their derivation. The net-value-added estimates indicate that 
participants in the research sample produced output worth an average of 
$293 per month of active participation during Phase 1 and $503 per month 
during Phase 2. Given an estimated average length of active participation 
of 5.5 months in Phase 1 and 3.8 months in Phase 2, these figures imply a 
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total value of output, discounted to the time of enrollment, of $3,434 per 

participant (an amount that would more than offset participants' in-program 
compensation). 

2« The Value of Out-of -Program Output 

STETS affected output produced outside of the program in two 
ways. As participants entered STETS, they gave up some alternative 
employment opportunities. Uter, as they completed their STETS training, 
many participants were able to work more than would have been the case in 
the absence of STETS. These changes in output enter the benefit-cost 
analysis from the perspectives of society and participants. 

Participants will perceive foregone production as a cost and 
increases in production as benefits. These changes also enter the social 
perspective to the extent that they represent a net change in total 
output. It is generally assumed that this is the case. However, if 
participants displaced workers who would have otherwise held the jobs 
filled by participants, then the lost output of those other workers must be 
subtracted from the increased output of participants in order to calculate 
the net change in social resources. In the extreme, STETS may have simply 
shuffled workers among a fixed number of jobs, with no net increase in 
output. The participants would have had higher incomes, but at the expense 
of other workers who were displaced. 

At the other extreme, STETS may have enabled participants to move 
from a labor market in which an excess supply of labor existed to one in 
which an excess of demand existed. When participants leave the market in 
which an excess of supply exists, any jobs that they would have obtained 
are filled by workers who would have been unemployed otherwise. From the 
social perspective, this effect implies that no output is foregone by 
having participants enter STETS, and thus that the social cost of 
participation is zero. In this case, social benefits would equal the 
increase in participant earnings plus the increased earnings of 
nonparticipants who fill jobs vacated by participants. Of course, this 
result requires that participants not be placed back into the excess labor- 
supply market. 
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STETS jobs often seemed to be in markets in which an excess supply 
of labor existed. The demonstration was fielded during an economic 
recession, and the possibility of displacement seems relatively high, both 
on the jobs that they held and on those that they would have held. These 
indirect labor-market effects will affect the social value of the output 
produced in Phase 1 and Phase 2, as well as the social value of the output 
produced after leaving STETS. 

The net change in social output in the presence of displacement can 
be valued in several ways. We chose to estimate net present value under 
the assumption that no displacement occurred, and then to assess the 
importance of this assumption by calculating alternative estimates under 
the assumption of some displacement. This approach can be thought of as 
indicating the potential of STETS to increase social output, provided that 
macroeconomic conditions are adjusted to take advantage of it. Moreover, 
given the absence of any empirical basis for estimating the extent of 
indirect lator-market effects (or even knowing the direction of their net 
effect), no clearly preferable alternative exists. 

Under these assumptions about indirect labor-market effects, the 
change in out-of-STETS output was estimated as the change in total 
compensation received by participants (i.e., gross wages plus fringe 
benefits). If the markets function competitively, then the actions of 
employers and workers will ensure that total compensation equals the value 
of workers' contribution to output. In addition, regulations pertaining to 
wage rates in sheltered workshops require that compensation in that sector 
reflect productivity. To estimate total compensation, we multiplied the 
after-tax earnings estimates derived from interview data by a factor that 
reflected tax-n/ithholding and fringe-benefit rates. This factor indicates 
that total compensation is 45 percent larger than after-tax earnings.^ To 
ensure that the estimates correspond to out-of-program output, we used non- 
STETS earnings as our measure. 



1 

As described in Appendix C, gross earnings (i.e., before tax 
withholding) are 23 percent greater than after-tax earnings. In addition, 
fringe benefits for low-^age workers such as STETS participants were 
estimated to be 18 percent of gross earnings. 
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The resulting calculations indicate that at the 6~month point, when 
most experimentals were in STETS, the controls had $16.89 more per week in 
non-STETS total compensation than did experimentals. This differential was 
reversed at month 15, when experimentals earned $5.76 more total compen- 
sation per week than did controls. At month 22, when all experimentals had 
left STETS and all earnings were from non-STETS employment, the experi- 
mental-control differential in total non-STETS compensation had risen to 
$13.50 per week. 

These estimates, along with the interpolations, are shown in Figure 
VIII. 1 (a zero experimental-control difference was assumed for the time of 
enrollment). The cumulative change in total compensation— which equals 
the value of out-^of-program output—is shown as the shaded area. It 
indicates that, during the fifteen months after randomization, participants 
forewent non-STETS jobs in which they would have produced output worth $437 
per participant. During the seven months between month 15 and month 22, 
participants produced increased non-STETS output worth $290 per partici- 
pant. When discounted to the time of enrollment, these estimates imply 
$425 of output foregone per participant in the first fifteen months and a 
subsequent increase in output worth $268 per participant. 

We have used these two figures to approximate both the foregone 
output while participants were in STETS and the increased postprogram 
output. Of course, since most participants spent less than fifteen months 
in STETS, this approximation is fairly rough. However, despite the 
imprecision in this disaggregation, the sum of the two estimates is an 
accurate estimate of the net change in non-STETS output during the 22-month 
observation period. 

D. OTHER BENEFITS OF STETS 

While the primary objective of STETS was to increase employment and 
earnings, the intervention also generated other important impacts. We 



1 

Details on the calculation of total compensation and the 
interpolation are provided in Appendix C. 
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FIGURE VIII. 1 



CUMULATIVE IMPACT ON TOTAL COMPENSATION 
PER PARTICIPANT FOR THE 22-MONTH OBSERVATION PERIOD 
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consider these impacts within the context of three groups: (l) changes in 
the use of programs other than STETS. (2^ changes in the use of residential 
programs, and (3) changes in government transfers and taxes. 

In general, these impacts were estimated by multiplying the 
estimated impact on months of use by a "shadow price," which is an estimate 
of the average monthly cost of use. For example, changes in the use of 
sheltered workshops by participants were valued by multiplying the 
estimated change in the number of months that participants used sheltered 
workshops by the average monthly cost of operating sheltered workshops. 
The changes in use were based on interpolation procedures similar to thoje 
used to estimate the change in non-STETS earnings. The monthly cost 
estimates were based on published data on the costs of the various 
programs. As mentioned in Section A, all benefits were valued in 1982 
dollars to correct for the effects of inflation. Thus, the monthly cost 
estimates used to value changes in use reflect 1982 dollars. 

^' The Use of Training a nd Service Programs Other than STETS 

Participants were expected to alter their use of training and 
service programs other than STETS. As was shown in Table VIII.l, these 
alternative programs included sheltered workshops, work-activity centers, 
schooling, job-training programs, case-management services, counseling! 
social/recreational programs, and transportation programs. In many cases,' 
particularly for sheltered workshops, STETS was observed to reduce the 
average use of these programs, but it could have increased use for some 
persons who needed to supplement their STETS services. Any changes in the 
use of these other programs consume or free up resources. Thus, such 
Changes are social costs or benefits. These changes also enter Into the 
nonparticipant perspective because that group funds these programs. 

Table VIII. 2 presents the estimated changes in the use of four 
types of alternative programs: (1) sheltered workshops,^ (2) secondary- 



1 



Because few sample members actually used work-activity centers we 
combined workshops and activity centers into a single category. 
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TABl£ VIII.2 



ESHMMED ynSE OF BENEFITS PER PARnCFAKT PROM REDUCED 
USE OF TRAINDC AND SERVICE PROGRAMS 
ODCR im STETS 



Prograro 



Estimated Change In Months of Use 
Months 1-6 Months 7-15 Months 16-22 



Estimated Estimated 

Value Per Benefit (Total 

Month Discounted Value 

(1982 Dollars) In 1982 dollars)^ 



Sheltered Workshop 

Secondary Vooational 
Special Education 



^.19 
-0.20 



Other School -0.11 
Other 3ob-Tralnlng Programs -0.99 
Total Benefits 



^.72 
^.^^2 

^.25 
-2.09 



-O.TZf 
-0.2^^ 

-0.0^ 
-0.92 



^^93 
520 

290 
113 



767 
112 



NOTE: Details on the derivation of the cumulative inpact estimates and the estimates of average value per 
month for the programs are presented in Appendix C. 

a 

Because reductions In use are classified as benefits, the discounted values are listed as positive 
niwbers. All dollar values are expressed in 1982 dollars and are discounted to the time of enrollment 
using a 5 percent real annual rate. 
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level vocational programs, (3) other schooling, and (4) alternative job- 
training programs (other than schooling or sheltered workshops). We 
estimated reductions in the use of all these programs over the 22-month 
observation period, with the largest reductions observed for alternative 
training programs and sheltered workshops. When these changes are valued 
and discounted to the point of enrollment, the value of the changes in 
program use is estimated to be $1,741 per participant. Thus, savings from 
the reduced use of alternative training and service programs over the first 
22 months offset approximately 30 percent of the operating costs. 

2. The Use of Residential Programs 

Most participants lived with their parents or relatives and 
continued to do so over the 22-month observation period. Thus, only small 
changes occurred in the use of residential programs. We estimated that 
slight reductions occurred in the use of institutions, group houses, and 
foster care; a small increase occurred in the use of semi-independent 
residential programs (e.g., supervised apartments), particularly at the 15- 
and 22-month observation points. The discounted net value of these 
observed changes in the use of residential programs ie $149. 

Our estimates of the impacts on living arrangements may fail to 
capture the longer-run effects of STETS. The STETS population is fairly 
young, and most sample members lived with their parents or relatives. This 
situation may not continue in the long run. Certainly, most young adults 
in the age range of the STETS target group would be moving into more 
independent living arrangements. Whether the employment and income impacts 
of STETS will induce participants to make such moves or prompt them to move 
into particular types of supported arrangements is uncertain. This 
potential long-run impact should be kept in mind when assessing the 
observed benefits and costs of STETS. 

3» The Use of Government Transfer Programs and Taxes 

We examined the use of three types of government transfer 
programs: SSI and SSDI, other cash transfers, and Medicaid and Medicare. 
For each program, we estimated the change in benefit payments (which 
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represent transfers between nonparticipants and participants) and the 
changes In program administrative costs (which represent the social and 
nonparticipant resource costs associated with making the transfers). 
Changes in benefit payments for SSI/SSDI and other transfer programs were 
estimated directly from interview data (interview self-reports were 
adjusted to reflect 1982 dollars). Changes in Medicaid/Medicare benefits 
were based on estimated changes in coverage (based on interview data) and 
on the average annual expenditure per disabled Medicaid recipient. 
Administrative costs for all programs were based on changes in transfer- 
program use and on published data on average administrative costs. 

As shown in Table VIII. 3, the largest savings were generated from 
reductions in SSI and SSDI. Medicaid/Medicare savings were also 
substantial. Together, the net savings from reduced transfer payments were 
estimated to be worth $545 per participant (after discounting). The 
corresponding administrative cost savings were estimated to be $31 per 
participant. 

Tax payments by participants are also viewed as transfers. An 
increase in tax payments is a cost to participants, but represents an 
offsetting benefit to nonparticipants (i.e., all other taxpayers); hence, 
they are transfers and do not enter into the social perspective. 

To estimate paid taxes, we used an estimate of the change in 
participants' total income and an estimate of the overall tax rate 
applicable to low-income households. As estimated by Pechman (1985), this 
tax rate was approximately 23 percent of total income, defined to Include 
earnings, public transfers, training allowances, etc. The major components 
of this tax rate are payroll, sales, and excise taxes. These taxes are 
difficult to avoid, especially those levied on consumption. Thus, even 
though participants generally faced low tax rates on earnings and may in 



1 

As is the case with all transfers, changes in the resource costs 
of making the transfers should be included in the social perspective. 
However, in terms of tax payments, the changa in administrative costs is 
probably very small and is treated as zero. Moreover, this treatment of 
taxes ignores the additional benefits accruing to participants because 
their tax payments support governmental activity. 
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TABUE VIII.3 



ESUmU) V^IC OF BDCHTS PER PARTICPANT FROM REDUCED 
USE OF TR^^SFER PROGRAMS 



Estimated Changes^ 



Months 1-6 Months 7»15 Months 16-22 



Estimated Estimated 

Value Pfer Benefit (Total 

Month Discounted Value 

(1982 Dollars) In 1982 Dollar.n^^ 



Trawfbr B^WBrt t s 

SSI/SSDI 

Other Welfare and 
Cash Transfers 
Medlcald/Medlcare 



AUnlstratiie Costs 

sa/ssDi 

Othei' Welfare and 
Cash Transfers 
Medlcald/Medlcare 



Total Benefits 



-31.30 


-125.80 


-121.9 


1 


-23,90 


-68.10 


7.8 


1 


-0.19 


-0.52 


-0.27 


215 




^.55 




13 


-0.25 


-O.W 


0.08 


11 


^.19 


^.52 


-0.27 


11 



264 
82 

200 

545 

n 

7 
10 
31 

576 



NOTE: Details on the derivation of the oimilatlve Impact estimates and the estimates of average value per 
month for tte programs are presented In Appendix C. Details do not dways sun to totds because of 
rounding. 

a 

For transfter payments from SSL/SS>1 and other welfare, the charges are in dollars of payments received. 
For adnlnistratlve costs for SSI/SSDI and other welfare, charges are In months of use. Changes In 
Medlcald/Medlcare are in months of coverage. 

b 

Because reductions in use are classified as benefits, the discounted values «e Usted as positive 
fwnbers. All dollar values are expressed in 1982 dollars and are discounted to the tiro of enrollment 
using a 5 percent real annual rate. 
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fact have avoided paying some payroll and income taxes, their total tax 
burden (as a percentage of income) vas not significantly different from the 
tax bvrden of most taxpayers (although the composition of taxes did vary 
considerably by income level). 

The change in total income was calculated from estimates of the 
changes in gross earnings and transfer payments > which is essentially the 
same measure of income used in Chapter VII, with the addition of an imputed 
value of tax withholding to change from after-tax to gross earnings. 
Altogether, we estimated that participants paid an average of approximately 
$250 more in taxes during the 22-'raonth observation period. 

E. OVERALL ASSESSMENT OF BENEFITS AND COSTS 

In assessing STETS as an investment, we first aggregated the 
measured benefits and costs described in the previous sections. We then 
considered the intangible (unmeasured) benefits and costs pertaining to 
changes in employment, social integrate a, and independence. Thus, the 
final assessment of STETS reflects both the measured and unmeasured impacts 
and costs. 

1 • Measured Benefits and Costs 

The individually measured benefits and costs are added together in 
Table VIII. 4. The estimates suggest that STETS created a net cost to 
society during the 22-month observation period. The measured social costs 
totaled $6,232 per participant, i^hile measured social benefits (increased 
output produced by participants and the reduced use of other training, 
service, residential, and transfer programs) totaled only $5,199 per 
participant. Participants clearly benefitted from their participation, 
receiving in-program compensation that more than offset their tax payments 
and their reduced use of transfers. Nonparticipant taxpayers incurred the 
costs both for operating STETS ana for participant compensation. They 
receiv^id substantial benefits (primarily from the increased output produced 
by participants in STETS and the reductions in their use of sheltered 
workshops, other job-training programs, and transfer programs), but these 
benefits offset only two-thirds of the costs incurred by nonparticipants. 



ERIC 



160 

214 



TABLE VIII. 4 



ESTIMATED BENEFITS AND COSTS OF STETS PER PARTICIPANT DURING THE OBSERVATION PERIOD, BASIC ESTIMATES 

(1982 dollars) 



Component 



Social 



Analytical Perspective 



Participant 



Nonpartldpant 



!• Program CoeU 

Project operations 
Payaents to participants 
Central administration 

II. (^tpot Pr^ttc«d by Ftertldpttnts 
Phase 1 and Phase 2 output 
Foregone output while In STETS 
Increased out-of-prograa output 

III. Other Prograas 

Reduced use of : 

Sheltered workshops 
Secondary vocational school 
Other school 
Job-training programs 

IV. Residential PrograM 

Reduced use of : 
Institutions 
Group homes 
Foster homes 

Semi-Independent residential 
programs 

V. Transfer Flsjments and Taxes 

Reduced SSI/SSDI 
Reduced other welfare 
Reduced Medic a id /Med lea re 
Increased taxes 

VI. Transfer Adstniatratlon 

Reduced use of SSI/SSDI 
Reduced use of other welfare 
Reduced use of Medlcald/Medlcare 

VII. IntaxKiblea 

Preferences for work 
Increased self-sufficiency 
Increased variation in 

participant income 
Foregone nonmarket activity 
Increased Independent living 



-$6,050 
0 

-182 



3,434 
-425 
268 



767 
428 
112 
434 



174 
72 
7 

-114 



16 



12 



+ 
+ 



$0 
3,094 
0 



0 

-425 
268 



0 
0 
0 
0 



0 
0 
0 
0 



-264 
-82 
-232 
-249 



0 
0 
0 



+ 
+ 



-$6,050 
-3,094 
-182 



3,434 
0 
0 



767 
428 
112 
434 



174 
72 
7 

-114 



264 
82 
232 
249 



16 
8 
12 



+ 
+ 



Met Present Valoa (Benefits less Costs) 



-$1,038 



$2,111 



-$3,149 



NOTE: Benefits and costs are discounted to the time of enrollment using a 5 percent real annual 
discount rate. 
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Although costs exceeded benefits for the observation period, trends 
in the impacts on earnings and the use of sheltered workshops and other 
job-training programs suggest that benefits will outpace costs beyond the 
observation period. Twenty-two months after randomization, measured 
benefits appeared to be exceeding costs at an annualized rate of almost 
$1,800 per participant (when discounted to the time of enrollment). Thus, 
as shown in Table VIII. 5, measured social benefits will exceed measured 
social costs if the benefits observed at month 22 continue undiminished for 
as little as seven months. This pattern of results reflects the investment 
nature of STETS. The intensive training and services are provided in the 
expectation that long-term impacts will be generated. Thus, almost all 
the costs are incurred up front, while the benefits accrue gradually over 
time. This pattern of benefits and costs was also observed in the national 
Supported Work demonstration for the AFDC target group (Hollister, Kemper, 
and Maynard, 1984) and in the Job Corps program (Mallar et al., 1982) — two 
training programs that were judged successful from a social benefit-cost 
perspective. 

Of course^ this result depends on participants' maintaining their 
regular jobs. The stability of job-holding for sample members who had jobs 
appeared to be high between the 15- and 22-month interviews. Thus, it is 
reasonable to assume that the increases in earnings and the reductions in 
the use of other programs will persist for some time. However, it is 
unknown whether the participants will be able to maintain their employment 
in the long run. If they fail to maintain their jobs, then the benefit- 
cost assessment could change. For example, if participants' gains in 
earnings and regular job-holding decay linearly to the levels for controls 
over the year following the observation period, then social benefits will 
fall short of social costs. Therefore, it is essential that the increased 
levels of regular job-holding and earnings be maintained in at least the 
short-run if STETS is to be considered a socially efficient investment. 



1 

If linear decay rates are assumed, then the experimental-control 
differentials in earnings and the use of other programs must not fall to 
zero until at least 15 months after the observation period if social 
benefits are to exceed social costs. 
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TABLE VI II. 5 



ESTIMATED SOCIAL BENEFITS AND COSTS PER PARTICIPANT IF 
OBSERVED IMPACTS CONTINUE FOR SEVEN MONTHS 
(1982 DOLLARS) 



Cowponent 



Total Benefits (Discounted to the Time of Enrollment) 
Months 1>22^ Months 23-29^ Total 



Program Costs $-6,231 

Output Produced by Participants 3,277 

Other Training Programs 1,7^^1 

Residential Programs 139 

Administrative Costs of 

Transfer Programs 37 

Net Present Value (Benefits $-1,038 
less Costs) 



SO 
3b^^ 
572 
103 



$1 ,0^^2 



$-6,231 
3,641 
2,313 
2^^2 

_JfO 
$4 



This period Includes the demonstration observation period. 

b 

Impacts are assumed to persist for this entire seven-month period at the level that was 
observed at the 22-month Interview. 
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For participants, STETS is likely to continue to represent a good 
investment. Because participants should continue to accumulate earnings 
increases (compared with what would have been the crse in the absence of 
STETS), their net benefits should increase beyond the observation period. 
Even if they fail to maintain their jobs and their earnings levels fall to 
those observed for controls, the increases while they were in STETS and 
during the immediate postprogram period will generate a net gain. 

For nonparticipants, STETS will continue to represent a net 
investment in participants, unless the effects observed at the 22-month 
interview persist for at least 2.5 years after the observation period. If 
the impacts decline, then the pay-back period (the time until non- 
participant net present value equaled zero) will be even longer. While Lt 
seems possible that impacts will persist for a sufficiently long time, we 
have little empirical evidence to support such a scenario at this time. 

The estimated values of the benefits and costs are based on sever.^'* 
assumptions and estimates. While these underlying assumptions and 
estimates are plausible, they introduce unavoidable uncertainty into the 
benefit-cost assessment. To evaluate this uncertainty, we examined 
alternative benefit and cost calculations that incorporated different sets 
of assumptions. The general procedure was to change one underlying 
assumption while keeping all others the same. These alternative estimates, 
which are presented in Appendix Table C.5, suggest that the overall 
conclusions presented above will hold as long as the operating costs can be 
kept to the levels observed during the steady-state period, and as long as 
the output produced during Phase 1 and Phase 2 has a value close to the 
value estimated in our case studies. 

The operating costs are clearly a key factor in the perspectives of 
society and nonparticipants. If these costs rise, STETS will appear to be 
less efficient from both perspectives. If we make the extreme assumption 
that the appropriate cost estimates should reflect the costs that were 
incurred during the entire demonstration rather than just those that were 
incurred during the steady-state period, the operating costs (discounted to 
the time of enrollment by using a 5 percent real annual rate) will rise by 
$2,500 per participant, yielding measured social costs that would exceed 
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measured social benefits by $3,552 per participant for the observation 
period. Under spch a circumstance, It Is uncertain whether Impacts will 
persist long enough to yield a positive net present value. 

Changes In the manner In which the value of ln~prograra output Is 
calculated can also affect the results* In our analysis, we measured the 
value of In-program out^^ut by using estimates of the net supply price. 
These estimates were based on data collected In case studies of 33 experl- 
mentals. They reflect our estimates of the net value added by partici- 
pants, and they assume that the value Is Indicated by the cost of having 
regular workers produce the output produced by participants In Phase 1 and 
Phase 2. As an alternative, we could have used the amount actually paid 
for the goods and services produced by participants In Phases 1 and 2. 
Because the STETS projects did not actively pursue revenue-generating 
policies, this alternative procedure yields a lower-bound estimate of the 
value. Because revenue was only 35 percent of our estimate of net supply 
price, using this alternative measure of the value would have reduced 
estimated benefits for society and nonpartlclpants. The resulting social 
benefits would have fallen short of social costs by $3,280 per participant 
for the observation period. For nonpartlclpants, costs would have exceeded 
their benefits by almost $5,400 per participant. Again, It Is uncertain 
whether future benefits would make up this difference between measured 
benefits and costs. 

Strong Indirect labor-market effects could also affect the 
assessment of STETS. If STETS participants were placed In jobs that would 
otherwise have been filled by other workers, then the earnings gains of 
participants would overstate the true Increase in output. We chose to 
Ignore these effects in most of the analysis, focusing Instead on the 
potential of STETS to Improve participant employment. Moreover, the lack 
of any empirical basis for estimating indirect labor-market effects 
Inhibited an analysis. 



1 

Gross supply price equals the cost to the most likely alternative 
supplier of the output. Net supply price was estimated by subtracting from 
gross supply price an estimate of the additional training and supervisory 
costs imposed on the employee by the participant. 
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However, we can judge how our assessment would change if these 
indirect effects had been taken into account. As an example, we assumed 
that one-half of the jobs filled by participants would have been held by 
other workers, and that one-half of the jobs that participants forewent 
when they entered STETS were subsequently filled by other workers. These 
assumptions had little impact on the social value of out-of -program 
earnings for the observation period, because the net value of those 
earnings was small. However, they would have reduced the social value of 
future earnings increases by 50 percent. The indirect effects would also 
have reduced the social value of in-program output from $3,434 per 
participant to $1,717 per participant. Because these reductions would have 
been borne by the displaced nonparticipants, the net present value of STETS 
to nonparticipants as a group would have declined. In this case, non- 
participants would have perceived a net cost of STETS of approximately 
$4,8<.0 per participant for the observation period. Because participants 
would have been unaffected by indirect labor-market effects, the social 
perspective, which combines participants and nonparticipants, would have 
changed from a net cost of $1,038 under conditions of no indirect labor- 
market effects to a net cost of $2,677 under the alternative assumptions. 
Thus, if indirect effects existed, impacts would have to persist longer 
into the future in order to generate social benefits that would exceed 
social costs. Under the assumption of our example, we estimate that the 
impacts observed at the 22-month interview would have to continue for two 
years for social benefits to outweigh costs (recall that benefits other 
than earnings are not affected by displacement). We conclude that, even 
with substantial indirect 3abor-market effects, STETS can be an 
economically efficient investment. 

The qualitative conclusions of the benefit-cost analysis were not 
altered by changing other assumptions about costs, the value of in-program 
output, and the appropriate discount rate.^ Changes in the linear interpo- 



1 

If we had used the OMB-mandated real annual discount rate of 10 
percent, social net present value for the observation period would have 
been -$1,068 per participant. Net present value for participants would 
have been $2,060 per participant, and would have been -$3,128 per 
participant for nonparticipants. These numbers are less than 3 percent 
different from those presented by using the 5 percent real annual rate. 
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lations underlying the estimates of cumulative effects would clearly have 
affected the estimates of benefits and costs. However, our overall 
findings should not be affected unless (1) the pattern of results was 
extremely different from the estimated pattern or (2) the 15- and 22-month 
points-in-time were unrepresentative of actual impacts. Neither of these 
situations seems plausible. 

2. Intangibles 

STETS services were intended to enhance the economic and social 
self-sufficiency of participants. In achieving this goal, STETS generated 
the measured impacts discussed in the preceding section — increases in 
earnings and regular job-holding, and reductions in the use of sheltered 
workshops and other training programs. However, STETS also generated 
intangible impacts by changing the quality of life for participants. The 
increases in earnings and the social integration of participants were 
expected to increase their satisfaction, as well as the satisfaction of 
nonparticipants who want to expand the opportunities available to mentally 
retarded young adults. Such intangibles cannot be measured accurately, but 
they are nevertheless important components of an overall assessment of 
benefits and costs. 

The measured impacts indicated that STETS did affect the activities 
and opportunities that were expected to generate intangible benefits. The 
increased income and the increased job-holding in the regular labor market 
should provide participants with benefits that go beyond the measured 
changes in output produced. However, in our analysis, we found limited 
evidence of changes in such intangibles as self-sufficiency and 
independence. In part, such limited evidence reflects tae inadequacies of 
our measures and the difficulty in measuring these concepts. It may also 
mean that self-sufficiency responds slowly to changes in opportunities, 
particularly for mentally retarded young adults. These persons may feel 
that they must maintain their jobs and their increased social interactions 
for a considerable time before they alter their behavior in terms of 
residential situation, benefactors, and financial independence. Finally, 
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we have no measures of any overall increases in satisfaction, other than 
the fact that many participants appeared to remain voluntarily in their 
jobs. 

Changes in earnings can also create intangible costs. For example, 
the stress of employment may create health problems. Moreover, persons who 
lose their eligibility for transfers such as SSI or SSDI face p>otentially 
more uncertainty about future income than had they remained in those 
programs. However, it is presumed that participants must value the 
increased earnings and interactions by more than the costs of the 
intangibles, because they made the choice to enter STETS, and because many 
continued in their jobs after leaving STETS. 

F. CONCLUSIONS ABOUT STETS AS AN INVESTMENT 

The benefit-cost analysis suggests that STETS can be a worthwhile 
social investment. From the perspective of society as a whole, it cost 
$6,200 per participant to provide the STETS intervention. During the 22- 
month observation period, this investment yielded increases in participant 
output and reductions in the use of other programs by participants that 
offset about 85 percent of this initial investment. Consequently, we 
observed a measured neu social cost of $1,038 per participant for the 
period covered by our data. However, trends observed for the impacts on 
earnings and the use of sheltered workshops suggest that benefits will 
persist and are likely to outweigh costs in the long-run. If the earnings 
and reduced alternative program benefits continued for as little as seven 
months beyond the 22-month point, social benefits would exceed social 
costs. This finding is consistent with other successful training programs 
that incur substantial up-front costs for training in order to generate 
long-run benefits. 

In the longer-run, it is possible that benefits would substantially 
outweigh costs. As indicated in Appendix C, we estimate that social net 
present value would exceed $5,200 per participant (a benefit-cost ratio of 
1.8) in ten years, even if the effects diminished by 50 percent every five 
years. This net benefit would be split between participants and 
nonparticipants, so that both groups would benefit from STETS. This 
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finding indicates the potential of STETS, but without longer follow-up 
results the ultimate magnitude of net benefits is uncertain. 

In addition to the measured benefits and costs included in the net 
present value estimates, the investment in STETS creates intangible 
benefits by increasing the social and employment opportunities available to 
the participants. While evidence is limited, the fact that many 
participants tended to remain voluntarily on their jobs suggests that STETS 
enhanced the quality of their lives. Furthermore, the increased income, 
regular job-holding, and social interaction provided by STETS are also 
expected to be benefits. 

Because of their increased earnings, both during and after STETS, 
participants benefitted substantially from STETS. On average, they 
received a net benefit during the 22-month observation period of $2,111 per 
participant, and this amount is expected to grow as they continue to work. 

For nonparticipant taxpayers, STETS represented a net cost during 
the observation period. This group paid not only the operating costs, but 
also the in-program participant compensation expenditures. Approximately 
two-thirds of these costs were offset by output produced by participants 
while in STETS, by savings from the reduced use of sheltered workshops and 
other programs, and by savings in transfer payments (primarily SSI/SSDI). 
If these impacts persist at their 22-month levels, then the nonparticipant 
costs will be recouped entirely within four and a half years after 
randomization. Nonparticipants may also receive intangible benefits from 
SXMS because it effectively achieves a widely stated public policy goal by 
increasing the employment opportunities and performance of mentally 
retarded young adults. 
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ERIC 



The results described in the previous chapters indicate that STETS 
was able to improve the emplojnnent prospects for mentally retarded young 
adults. It appeared to increase regular job-holding and total earnings, 
while generating reductions in the use of sheltered workshops. It also had 
small overall effects on the use of transfer programs. All in all, the 
program appears to be a success and will represent an economically 
efficient social investment if costs can be kept at or below the 
demonstration levels, and if impacts persist for as little as one year 
beyond the 22-month observation period. 

In this chapter, we first summarize the main evaluation findings on 
the impacts of the program. Then, in Section B, we review the results of 
the benefit-cost analysis. Finally, Section C presents some comparisons 
with the results of other programs and summarizes our overall policy 
conclusions. 

A. ESTIMATED PROGRAM IMPACTS 

The analysis considered the impacts of STETS on four major areas: 

(1) labor-market behavior, (2) training and schooling, (3) public transfer 

use, and (4) economic status, independence, and life-style. Key findings 
are summarized in Table IX. 1. 

!• Labor Market Behavior 

The evaluation clearly indicates that a STETS-type program can be 
expected to improve the postprogram employment prospects of mentally 
retarded young adults. As shown in Table IX. 1, employment in regular jobs 
was significantly greater for experimental group members than for control 
group members in the postprogram observation period—that is, at months 15 
and 22. By month 22, experimentals were an average of 62 percent more 
likely than controls to be employed in a regular job (31 percent versus 19 
percent), and the regular-job earnings gains were proportionately larger 
($36 per week among experimentals versus $21 per week among all controls). 
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ESTimTED PnOCRAN I>f AaS ON KEY OUTCOC ^CASUFCS 



Month 15 



Out cone HMaurat 

Rsroert G^ployvd In 
RegjUr W 

RsrOBrtt Ej^pbyvd In 
AnyP«ld Obb 

Avengt tPedcly Earnings 
In RagjUr Oab 

Average Itoddy Earnings 
in My Paid Oob 

Tralfiiiig Md SdiMllng 

Rsroent In Any Training 
Reromt In Any Schooling 



Coitrol Control Control 

ExperUerttal Qrop EstUitted ExperlMental Qraup EstlMted ExperlMerttal Group Estlmted 
(ktsup Hem Ma*^ Inpact Qroo Mean Mean Lipact Qroup Mean Moan Impact 



Peroart Receiving S9 
or SSDI 

Average Monthly IhcGHB 
ftmSaorSSDI 

Reroert Receiving Any 
Cash Trmfers 

Average Monthly Incone 
fcm Caeh Transfers 



11.8 10.7 1.1 

«7.8 45.2 

$11.81 $9.81 $2.00 

$ 52.39 $ 25.93 $ 26.^»» 

61.7 *0.6 21.1»» 

7.5 15.7 -8.2« 

».3 31.0 -4.7 



31.7 



as.2 16.8 9.4»» 31.0 19.1 11.9« 



44.8 43.6 1.2 



44^7 43.7 



1.0 



$ 26.90 $16.31 $10.59^ $ 36.36 $ 20.55 $ 15.81»» 



$37.91 $26.48 $ 11.43»» $40.79 $28.41 $ 12.38** 



2D.6 28.4 -7.8» 

6.2 10.1 -3.9 

33.1 40.7 -7.6»» 



16.6 29.1 -12.5»» 

8.0 11.4 -3.4 



34.9 40.2 



-5.3 



$ 66.41 $ m.59 $ -8.18 $ 91.35 $ 109.65 $ -«.» $ 99.27 $ 120.03 $ -30.76 



43.1 -11.4»» **.5 51.5 -7.0» 



52.0 -2.4 



$©.23 $ 99.96 $-19.75 $114.78 $ 138.72 $-23.94 $136.53 $ 136.08 $-9.55 



Average Veddy Pttsonal $71.72 $ 50.94 $ 20.78« $67.22 $ 59.67 $7.55 $71.59 $ 62.39 $9.20 



InoM*' 



NOTES These reaults were estlMted through ortttnary least sqyves techniques* EstlsMted iapects or selected binary and tnMted outccM 

MBuces mn generated using prottt ml toMt analycU, respectively* with virtually identical results. (These results re presented in 
Appendix A to the report.) 

^Iegul4r Jobs are those that are neither training/nodc-study nor eorfcshop/activity-oenter >obs. 

*Wrsonal incoM includes earAings» cash tremlWr beneflU (ATOC, general assistanoe, Suppl^erttal Security Inc<Me» and Sodal Seoiriiy 
Disability Ineuranoe), and other regular souroes Incow. 

•Statlstioslly signifiovt at the 10 percent level, tM>-talled test. 
•^StatUtically sigtificant at the 5 percent level, t>o*taUed test. 
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Furthernwre, as shown in Figure IX. 1, this postprogram increase in employ- 
ment in regular jobs was roughly equal to the reduction in employment in 
workshops and activity centers. The STETS experience tended not to affect 
the average postprogram incidence of holding non-workshop-training jobs. 
Thus, although the overall level of employment was largely unchanged, very 
important compositional effects occurred. Overall, average earnings from 
all types of employment increased by 44 percent Jn month 22 ($41 per week 
for experimentals versus $28 per week for controls). 

As shown in Table IX. 2, the postprogram (month 22) results vary 
substantially among subgroups of the sample. The St. Paul program clearly 
had the largest Impact on the regular job-holding of participants (41 
percent of the experimentals, compared with 18 percent of the controls, 
held regular jobs). In terms of the differential impacts among subgroups 
defined by their personal characteristics, STETS seems to have been most 
effective for four groups: those with lower IQ scores and whose retard- 
ation has organic causes; older individuals; males; and those who were more 
independent, as evidenced by their living arrangements and money-management 
skills at the time they enrolled in STETS. 

Of these results, two sets arc especially thought-provoking~the 
results by IQ level and the results by gender. Estimated program impacts 
on the probability of holding a regular job were essentially zero for those 
participants with borderline IQ levels, 12 percentage points for those 
whose IQ scores indicated mild retardation, and 28 percentage points for 
thccs vhcGc IQ scoico l-uvIioaLevl luuuejLdLe reLairdacion. ine net errect of 
these differential program results is that STETS tended to raise the 
employment prospects for the mild and moderate retardation subgroups from 
levels that in the absence of STETS would have been well below those for 
the borderline retarded group to levels that were roughly similar to the 
borderline group. 

The impact estimates for males and females show clear patterns 
which show that STETS had substantial impacts on the employment and earn- 
ings of males but no impacts on the employment and earnings of females. 
For example, both male and female participants would have earned about $18 
to $20 per week on regular jobs ($107 per week among those employed) in the 
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FIGURE IX. 1 



TRENDS IN TYPES OF JOBS HELD BY 
EMPLOYED EXPERIMENTALS AND CONTROLS 



Bas«lfn« 



TralnlTHi (472) 




EXPERIMENTALS 
Month 6 

Regular (18%) 



Training (72t) 




: 1 Workshop (lOX) 



Regular (18Z) 



Workshop (35t) 



CONTROLS 
Month 6 



Training (48:) 




Regular (24X) 



Workshop (28Z) 



Month 15 



f.^r^ 



Workshop (I IX) 




Regular (S8Z) 



Month 22 



Training (ihX) 



Workshop (ISX) 




Regular (693:) 



Training (29t)^ 



Month "5 




Regular (38Z) 



Workshop (331) 

Month 22 

Training (I4t) 



Workshop (42%) 
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NOTE: This figure is based on the regression-adjusted data presented in Table IV. 4. 
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Regular (44%) 



IX.2 



EsnmTS) HWCWM WAas a< key ourcacs n©(TY-m) months arer 

E>#WLliCKr FDR KEY SU5(mjPS OF STETS P/WTIdP/WS 



Subgroups Otfln«d 
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Bofderllne 
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Hoderats 
Abb 

Younger thm 22 
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Mile 
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Cbntrol 
Meg^ 
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Mea n Lupacts 





».1 






$120.03 


t7.9 


1.6 


16.3 


67.34 


105,07 


24.7 


9.3 


15> 


180.56 


207.39 


♦3.4 


32«2 


11.2 


65.42 


126.11 


41.1 


17.9 


23.2» 


49.65 


:;o.69 


29.6 


30.8 


-1.2 


93.74 


96.90 


34.1 


2B.7 


5.4 
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MDTEt These reailts mre eetlMated throigh ordinary least squares techniques. The controJ varlAles Included In the Models ihldi undterUe the 
overall net lapact estlMites are defined in Appendix A to tr» r«poct. 

^legjlar >abs are those that are neither tralnlngAiocfc-study nor •oriohop/actlvlty-oenter Jobs. 

•StatUtlcally slT^lficant at the 10 percent le%«l, teo-taUti test. 
♦•Statistically significant at the 5 percent levels teo-talled test. 
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absence of STETS, as evidenced by the earnings of the control group mem- 
bers. However, participating in STETS raised the average regular-job earn- 
ings of males by 144 percent, to $48 per week ($134 per week among those 
employed). The average vegular job earnings of females ii: the experimental 
group were comparable to those of their control-group counterparts in month 
22 (about $21 per week, on average). These differential results for males 
and females seem to be related to the observed greater difriculty of the 
programs in placing females in Phase 2 program jobs, especially during the 
non-steady-state periods of program operations. 

2. Training and Schooling 

The employment impacts that were estimated for ST£T^ were hypothe- 
sized to lead to reductions in the use of other training programs and 
schooling. Such an outcome is important because many forms of training and 
schooling for mentally retarded young adults are expensive, long-term 
programs that do not have a strong record of placing individuals into 
competitive employment. 

A negative program impact on schooling was evident at month 6 as 
the activities of experimentals were absorbed by STETS; a smaller negative 
impact persisted in the postprogram period. However, the pattern of the 
impacts on training is more complicated. Although controls used training 
programs to a great extent during the early months after their application 
to STETS, experimentals used such programs to an even greater extent during 
the in-program period, since STETS, itself, was a training program. How- 
ever, as expected, STETS did substantially reduce the use of training 
(primarily workshop-related training) in the postprogram period by an 
amount (43 percent) that was roughly equal to the postprogram increase in 
regular employment (see Table IX. \). Thus, the STETS-induced increase in 
employment did carry with it overall resource-cost savings from the reduced 
use of training programs and schooling. 

3. Public Transfer Use 

The hypothesized employment impacts were expected to lead to 
reductions in the use of public transfers. Although some evidence suggests 
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that STETS did reduce dependence on cash transfers, the results are smaller 
than were initially expected. The impacts shovm in Table IX. 1 are reason- 
ably large for the in-program period, with a 26 percent decrease in the 
receipt of aay cash transfers (32 percent for experimentals versus 43 
percent for controls). However, this impact faded over time, until no 
statistically significant impact was evident by month 22 (although the 
point estimate is a reduction of 3 percentage points in transfer use and an 
average of nearly $10 per month in benefits). 

It is noteworthy that relatively larger persistent reductions were 
estimated for the proportion of cash transfers accounted for by Supple- 
mental Security Income and Social Security Disability Insurance. (The 
percentage decrease in tha receipt of these transfers hoverec? around 15 
points.) In month 22, the $21 per-month average reduction in SSI/SSDI 
benefits offset about 40 percent of the average $54 per-month earnings 
gains; at the same time, benefits from other cash transfers (primarily 
welfare benefits) increased by 50 percent, from $19 to $29 per month, due 
at least in part to the increased receipt of unemployment insurance 
benefits. 

The subgroups that were responsible for the small overall reduc- 
tions in cash transfers are generally those that experienced reductions in 
SSI/SSDI income. As shown in Table IX. 2, these subgroups are often, but 
not always, the same subgroups that experienced the greatest gains in 
regular employment from participating in STETS. 

^« Economic Status, Independence, and Life-Style 

The impacts of the program on employment were also expected to 
Influence socio-economic status, independence, and life-style. STETS did 
raise the total personal income of participants, but by less than would be 
indicated by the earnings gains, since the earnings gains coincided with 
reductions in cash transfer benefits, as was noted above. In the post- 
program period, STETS increased total personal income by between $8 and $9 
(see Table IX. 1). Perhaps because of the modest level of this increase in 
personal income (generally among the same groups who were moving into 
regular employment, as shown in Table IX.2), we failed to detect any strong 
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patterns of program impacts in the postprogram period on measures of 
invdependence and life--style# Admittedly, measures of independence and 
life-^style are less well defined than are measures of the other outcomes 
under evaluation. However, it is likely that the income gains for most 
experimentals were too small to effect measurable changes in other aspects 
of independence or life-style; and for those with more substantial income 
gains, the observation period may simply have been too short to expect that 
the gains would translate into long-term adjustments in these areas* 

B* RESULTS OF THE BENEFIT-COST ANALYSIS 

The benefit-cost analysis suggests that programs such as STETS can 
be a worthwhile social investment. On average, it cost $9,400 per partici- 
pant to provide the STETS intervention, of which $3,100 represents wages 
paid to the participants in Phase 1 and Phase 2 jobs and so not a cost to 
society. During the 22-montb observation period, this investment yielded 
increases in output and reductions in the use of other programs by partici- 
pants that offset about 85 percent of this initial investment. Consequent- 
ly, we observed a measured net social cost of about $1,000 per participant 
for the period covered by our data. However, trends observed for the 
impacts on earnings and the use of sheltered workshops suggest that bene- 
fits will persist and are likely to outweigh costs in the long-run. If the 
earnings gains and benefits from the reduced use of alternative programs 
continued undiminished for as little as seven mDnths beyond the 22-month 
point, social benefits would exceed social costs. This finding is consis- 
tent with other successful training programs that incurred substantial up- 
front costs for training in order to generate long-run benefits. 

In addition, the STETS investment created intangible benefits by 
increasing the overall employment and social opportunities available to the 
participants. While the evidence is limited, the fact that many partici- 
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This average cost per participant compares fa^'orably with the 
average costs of other similar programs targeted toward jientally retarded 
populations and other disadvantaged groups (see Thornton and Maynard, 1985, 
for such a comparison). 
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pants tended to remain voluntarily on thexr jobs suggests that STETS 
enhanced the quality of their lives. Furthermore, the increased income, 
regular job-holdiiig, and social interaction provided by STETS are also 
expected to represent benefits. 

Because of their increased earnings, both during and after STETS, 
participants benefitted substantially from STETS. On average, they 
received a aet benefit during the 22-month observation period of $2,100 per 
participant, and this amount is expected to grow as the participants 
continue to work. 

From the perspective of nonparticipant taxpayers, STETS required a 
substantial investment of $3,100 per participant during the observation 
period. This group paid not only the program operating costs, but also the 
in-program participant-compensation expenditures. Approximately two-thirds 
of these costs were offset by output produced by participants while in 
STETS, by savings from the reduced use of sheltered workshops and other 
programs, and by savings in transfer payments (primarily SSI/SSDI). If 
these impacts persist at their 22-month levels, then the nonparticipant 
costs will be entirely recouped within four and a half years after random- 
ization. Even if measured benefits to nonparticipant taxpayers fall short 
of their costs, STETS may still be an attractive investment, because it 
effectively achieves a widely stated goal of public policy—to increase the 
employment opportunities and performance of mentally retarded young adults. 

C. OVERALL POLICY CONCLUSIONS 

The results of this evaluation demonstrate that many mentally 
retarded young adults can indeed perform adequately in competitive employ- 
ment situations, sometimes with minimal or no ongoing support services. 
More importantly, the results of this study indicate that transitional 
employment services such as were provided by STETS can be very instrumental 
in helping mentally retarded young adults achieve their employment 
potential. 
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These overall evaluation results compare favorably with employment 
and training programs targeted on other disadvantaged subgroups of the 
population. Most notably, the results compare favorably *^th those of the 
Supported Work demonstration, which was influential in the design and 
public support for the STETS demonstration. 

These results for the STETS program also strengthen the broad 
conclusions of the feasibility of implementing transitional and supported 
employment programs for mentally retarded young adults. More importantly, 
they provide evidence that implementation on a moderate scale (40 to 50 
slots) is feasible at an average cost per participant that is within the 
range of the cost estimates for both employment and training programs 
targeted on other segments of the disadvantaged population and alternative 
programs designed to serve mentally retarded persons. 

The specific policy conclusions that evolve from this evaluation of 
the STETS demonstration pertain to (1) the potential of STETS to mitigate 
the employment and independence problems of mentally retarded young adults; 
(2) key features of program design and targeting; and (3) the benefits and 
cos ts of the program. 

!• Program Potential 

STETS-type programs can be expected to have no impact on the 
overall employment level of mentally retarded young adults who are 
recruited through methods similar to those used in this demonstration. 
However, they will tend to move workers out of workshops and into 
competitive employment. While the net gain in unsubsidized employment 
attributable to STETS is about 12 percentage points, this overall result 
represents aggregate impacts and masks the substantially larger effects for 
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See further discussions of these cross-program comparisons in 
Thornton and Maynard (1985). Table A. 15 presents estimates of the average 
costs of STETS and alternative programs. 
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selected subgroups of the target population. Even at this apparently 
modest overall figure, the estimated impacts represent a movement into 
competitive employment of 60 percent of those who would otherwise be in 
workshops or activity centers. Furthermore, if one looks at the effects 
during the steady-state period of operations, when the programs success- 
fully served both males and females, the overall employment impacts are 
about 25 to 30 percent larger. Moreover, evidence suggests that employment 
retention will be quite high among those who are successful in making the 
transition to other employment. 

These programs can be expected to reduce public outlays for 
SSI/SSDI and education and training services substantially (especially 
sheltered workshops). However, partially offsetting increases in outlays 
for other cash transfer programs can also be expected. 

No clear evidence exists to indicate that aspects of life-style or 
social well-being will improve. However, since participation in this 
program was voluntary, one can assume that most, if not all, participants 
placed some nonpecuniary value on obtaining regular employment. 

2. Program Design and Targeting 

Several noteworthy results from this evaluation pertain to program 
design and targeting issues. First, evidence clearly suggests that on- 
going programs might well achieve substantially greater success than was 
observed for the full STETS demonstration, since we found much larger 
estimated employment impacts when we compared the employment behavior of 
experimentals and controls who partcipated in the program during the 
"steady-state" period — an approximately five-month period when project 
operations were relatively stable. Second, Phase 2 activities (the 
supported employment and crainlng experience within a competitive 
employment setting) were a key part of the program treatment. Third, 
programs should not "cream" in selecting participants. As has been found 
in evaluations of employment and training programs targeted toward other 
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segments of the population, STETS was often very successful with those 
whose prospects for entering competitive employment in the absence of the 
program w<?re the lowest. One noteworthy characteristic that identified 
such groups was an IQ score that indicated mild to moderate retardation. 

One caveat with respect to the targeting issue is the strikingly 
more favorable impacts of the program for males than for females, even 
though both groups are expected to have quite similar success rates in 
entering competitive employment in the absence of STETS. In view of the 
estimated importance of Phase 2 activities, the fact that a disproportion- 
ate share of female participants relative to males never entered Phase 2 
employment may explain, in part, the lack of significant employment impacts 
for females. Obviously, further investigation of the nature of the greater 
difficulties in serving females is warranted. However, the results of this 
study suggest that a major factor that affects the ability of program 
operators to meet the greater challenge presented by female participants is 
simply whether their attention is diverted by major program operational 
changes, such as start-up or phase-down activities; in the steady-state 
period, their success with females was only slightly lower than their 
success with males. 

3. Benefits and Costs 

In the long-run, STETS-type programs are probably a socially 
justifiable investment, in as much as the total benefits (the net increase 
in the value of the goods and services produced) will outweigh the costs of 
the program. However, as is generally the case, taxpayers must be willing 
to make a substantial up-front investment that may not be repaid to society 
for several years. Using the full STETS sample and assuming some modest 
decay (5 percent per year) in the postprogram results, we estimate that 
this pay-off period to society is about two and a half years, and the pay- 
off period to nonparticipants (tax payers) would be about four and a half 
years. However, if the effectiveness of ongoing programs were more similar 




182 



235 



to the effectiveness of STETS during its "steady-state" operations, this 
payback period could be substantially shorter. 

^. Generaliz ability of the Findings 

In view of the evaluation design, the results of this study are 
quite robust and provide reliable estimates of what one would expect upon 
replicating the STETS demonstration. However, the study findings are 
limited in two respects. First, they are based on only five, judgmentally 
selected urban sites, where programs were specially designed and 
implemented for this demonstration. We cannot be certain whether other 
program operators in other sites and who operate on-going programs under 
different social, political, and economic conditions would have similar 
experiences. (For example, economic conditions were generally poor during 
the period of this demonstration.) It is especially problematic whether 
similar programs could be efficiently and effectively operated in rural 
areas or even in more dispersed labor markets. Second, the participants 
represent only a small portion of the areas' populations of mentally 
retarded young adults, with most of the screening and selection having 
occurred at the social service agencies which made referrals to STETS. The 
evaluation design did not enable us to estimate or, more importantly, to 
characterize the selection processes and participation decisions. Thus, we 
are unable to predict the total portion of the target population that could 
potentially be moved into competitive employment through the provision of 
STETS-type services. 

Despite these limitations with the study, the overall conclusion 
that transitional employment is an effective program for increasing com- 
petitive job-holding among mentally retarded young adults is quite clear. 
In fact, this program appears to be among the more effective employment and 
training initiatives that have been field-tested. For example, these 
results compare favorably with the often-cited examples of successful 
employment and training programs (Supported Work for the long-term welfare- 
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dependent population and Job Corps), and they are much more favorable than 

2 

the results for youth-oriented initiatives other than Job Corps. 



1 

See Hollister et al. (1984) and Mallar et al. (1982). 

2 

See, for example, Maynard (1984), Bassi and Simms (1983), and 
Dickinson et al. (1984). 
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TABLE A, 1 



DEFINITIONS AND MEANS OF BASELINE CONTROL VARIABLES USED 

IN THE ANALYSIS 



Baseline 

Control Variables 



Reference Period 



Month 6 



Site 

Cincinnati ,205 

Los Angeles ,201 

New York ,244 

St. Paul ,127 

Tucson ,223 

IQ Level^ 

Borderline ,293 

Mild ,583 

Moderate ,124 

ige 

Younger than 22 ,731 

22 or older ,269 

Gender 

Male ,569 

Female ,431 

Sace/Ethnlcity 

Black ,318 

Hispanic ,109 

White and other ,573 

Ilvli^g Arraqgeaent 

Living with parents ,830 

Living in, supervised ,095 
setting" 

Living independently ,075 

Financial Nanageaent Skills^ 

Independent ,291 

Not independent ,709 

Kecelpt of Transfers 

SSI/SSDI . ,324 

Other transfers only° ,293 

No transfers ,383 

Secondaiy Handicaps^ 

Secondary handicap ,371 

No secondary handicap ,629 

Cause of Retardation 

Organic ,164 

Non-organic ,836 

Benefactor^ 

Benefactor ,286 

No benefactor ,714 

Work Experience In Two 
Tears Prior to Enrollaent 

Regular job lasting ,148 
?3 months 

OtTTer job lasting ,360 
>3 months 

OttTer ,492 



Month 15 



Month 22 



.207 
.214 
.239 
,124 
.216 



,294 
.599 
.107 



,692 
.308 



.577 
.423 



.321 
.144 
.535 



.823 
.095 

.082 



.274 
,726 



,335 
.321 
.344 



.358 
,642 



.163 

.837 



,291 
.709 



.147 
.348 
.505 



.198 
.213 
.237 
.129 
.223 



.289 
.600 
.111 



.689 
.311 



.565 

,435 



.311 
.142 
,547 



,815 
,101 

,084 



,274 

.726 



,336 

.329 
,335 



.354 
.646 



.171 
,829 



,296 
.829 



.147 
.342 
.511 
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TABLE A.l (continued) 



Baseline 

Control Variables 



Month 6 



Month 15 



Reference Period 



Month 22 



School Status at Referral 



Enrolled 



.304 
.696 




.310 
.690 



Not enrolled 



Bicperlaental Status 



Experimental 



Control 



.512 
.488 



.507 
.493 



*522 
.478 



Number in Sample 



283 



402 



395 



NOTE: All control variables are measured at baseline or referral. These 
means pertain to those sample members included in the Employment 
Outcome models. The following data items were obtained from the 
Application/Enrollment form, which was completed by the referral 
agencies: IQ level, age. gender, race/ethnicity, secondary handicap, 
cause of retardation, prior work experience, and school status at 
referral. The other data items were collected during the baseline 
interview. 

^The three classifications of IQ level are defined as follows: borderline 
is from one to two standard deviations (SD) below the mean on tests, mild 
is from two to three SD below the mean, and moderate is from three to four 
SD below the mean. IQ scores on the Wechsler scales (SD-15) which fall 
within these ranges are: borderline 70-84, mild 55-69, moderate 40-54. 
The ranges for scores on the Stanford-Binet test (SD«io) are slightly 
^-^^^^J^^^' ^^^^ sample members, only the range of retardation, and 

not IQ score, was report^-*.. The American Association of Mental 
Deficiency no longer reco/,nizes the borderline classification, but the 
term is used in this report to classify individuals with test scores 
above the mild range. 

^Supervised settings include institutions, group homes, supervised apartments, 
and other semi -independent settings. 

^Independence is defined as performing without assistance at least two 
financial management activities (shopping, handling bills, and using bank 
accounts), and without assistance in the other financial management activity. 

^Other transfers Include Medicare, Medicaid, AFDC, General Assistance, or 
Social Securi2:y received by or on behalf of the sample member, and food 
stamps received by the sample member *s household. 

^Secondary handicaps include: seizure disorders, visual impairment, emotional 
impairment, cerebral palsy, specific learning disability, mobility 
i}" ^™ others. Note that STETS clients were to have no secondary 

disability that would make on-the-job training for competitive employment 
impractical (see the discussion of the program eligibility criteria in 
Chapter II). 

^Benefactors are defined as individuals named by the primary sample member as 
providing assistance in two or more of the following areas: job search, 
residential counseling, financial management (including helping with the 
receipt of transfer payments), other counseling, and transportation. These 
individuals could be relatives or friends of the sample member, or service 
agency staff members. 
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TABLE A. 2 



ESTIMATED COEFFICIENTS IN MODELS USED TO GENERATE SELECTED 
NET IMPACT ESTIMATES REPORTED IN TABLE IV,4, IV,6, AND A,4 



Outcome Measure and Time Period 



Control Variables 



Earnings In Regular Jobs Percent With Any Paid Job 
Month 6 Month 22 Month 6 Month 22 



Intercept 

Site 

Cincinnati 
Los Angeles 
New York 
St. Paul 
Tucson 

IQ Level 

Borderline 
Mild 

Moderate 
Age 

Younger than 22 
22 or older 

Gender 
Male 
Female 

lece/Btlmicltj 

Black 

Hispanic 

White and other 

LiTlilK Arraqgenent 

Living with parents 
Living In supervised 

setting 
Living independently 



Financial Kanageai 

Independent 
Not Independent 



int Skills 



teceipt of Transfers 

SSI/SSDI 

Other transfers only 
No transfers 

Secondary Handicaps 

Secondary handicap 
No secondary handicap 

Cause of Retardation 

Organic 
Non'-organic 

Benefactor 

Benefactor 
No benefactor 

Work Experience in Two 
Tears Prior to Enrollment 
Regular Job lasting 

>3 months 
OtlTer Job lasting 

>3 months 
OtlTer 

School Status at Referral 

Enrolled 
Not enrolled 



7,1 



-12,4 

-19,1 
0.6 



7.3 
9.7 

6.2 



-0.4 



-1.3 
-1.2 



6.0 

-16.0 
7.4 



-4.3 
0.8 



-7.7 
13*6 
12.3 

1.6 
0.4 

-14.4 



37.7 



-44.1 
-43.4 

-31.6 
-14.5 



10.2 
-4.5 

15.9 
2.1 



-5.2 
8.7 



-5.9 
-4.1 
7.5 



-7.9 
-8.3 



-4.7 
-5.2 
1.5 

22.8 
16.1 

-13.7 



68.7 



-21.0 
-21.7 

-3.7 
-2.1 



11.9 
16.4 

-14.3 



6.8 



-11.2 
-15.5 



2.7 
-45.2 
2.3 



6.2 
-2.8 



-8.6 



1.9 



-6.6 



-3.0 
8.0 



-17.0 



0.7 



-40.6 
-23.2 

14.4 
3.8 



3.7 
17.4 

-6.7 
-5.9 



-12.3 
-11.1 



-6.6 
-35.4 
-2.9 



1.7 
-2.9 



-5.6 
-3.3 
3.0 

9.7 
21.3 

-6.5 



A. 3 



247 



TABLE A. 2 (continued) 



Outcome Measure and Time Period 

Earnings in Regular Jobs Percent With Any Paid Job 
Control Variables Month 6 Month 22 Month 6 Month 22 



Kxperlaental Status 

Experimental 
Control 

B*Site 

Cincinnati 
Los Angeles 
New York 
St. Paul 
Tucson 

S*XQ Level 

Borderline 
Mild 

Moderate 

S*Age 

Younger than 22 
22 or older 

F^Geoder 

Male 
Female 

B*t«ce/Ethiiiclty 

Black 

Hispanic 

White rnd other 

B*LlTiqg Arraqgeaeat 

Living with parents 
Living in supervised 

setting 
Living independently 



E*Financl«Kl Msoitges 

Independent 
Not independent 



tnt Skills 



E*Keceipt of Transfers 

SSI/SSDI 

Other transfers only 
No transfers 

B^Secondary Handicaps 

Secondary handicap 
No secondary handicap 

E*Cause of Ratardatlon 

Organic 
Non-organic 

E*Beiaef actor 

Benefactor 
No benefactor 

B*Vork Experience In TWo 
Tears Prior to Enrollment 

Regular Job lasting 

?3 months 
Otlier job lasting 

>3 months 
OtlTer 

B*Scliool Status at Referral 

Enrolled 
Not Enrolled 



18a 
30.7 

42.7 
3,7 



-5,0 
-14.0 

-18.1 



1K7 
6.2 



4U4 
-9.1 



-6.0 
-5.8 



-U.O 



12.7 
5.6 

14.8 



-5.9 



-0.4 
9.6 

12.2 
-14.6 



4.5 
17.2 

-17.4 



-28.6 



2.4 
12.9 



24.3 
23.3 
19.5 



20.5 
30.3 



10.1 



1.1 
-20.9 

4.6 



27.7 



-6.4 
-16.2 

-4.9 
19.1 



-10.9 
-22.2 

11.4 
-12.1 



10.6 
34.0 



-17.0 
35.0 
-4.8 



4.7 
14.0 



-8.1 



14.4 
-10.5 

5.8 



-23.0 



17.0 
2.7 

-2.2 
-2.8 



-2.2 
-3.0 

-3.2 



-11.4 



1.1 
29.4 



19.3 
23.2 
12.0 



5.1 
20.6 



15.0 



-1.0 
-18.6 



-2.9 



NOTE: These results were estimated through ordinary least squares techniques. All control variables 
are measured at baseline or referral. The variable definitions and their means are reported in 
Table A. 1. B* indicates that a control variable has been interacted with Experimental 
Status. The t-statistics for the coefficient estimates are reported in Table A* 3* 
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TABLE A. 3 



t-STATlSTICS FOR ESTIMATED COEFFICIENTS IN MODELS USED TO GENERATE 
SELECTED NET IMPACT ESTIMATES REPORTED IN TABLE IV. 4, IV. 6, AND A.4 



Outcome Measure and Time Period 



Control Variables 



Earnings in Regular Jobs Whether Any Paid Job 
Month 6 Month 22 Month 6 Month 22 



Intercept 

Site 
Cincinnati 
Los Angeles 
New York 
St. Paul 
Tucson 

IQ Level 

Borderline 

Mild 

Moderate 

Age 

Younger than 22 
22 or older 

Gender 

Male 
Female 

Sace/Ethnicity 

Black 

Hispanic 

White and other 

Living Arrangement 

Living with parents 
Living in supervised 

setting 
Living independently 

Financial Hanageaent Skilla 

Independent 
Not independent 

Receipt of Transfers 

SSI/SSDI 

Other transfers only 
No transfers 

Secondary Handicaps 

Secondary handicap 
No secondary handicap 

Cause of Retardation 
Organic 
Non-organic 

Benefactor 

Benefactor 
No benefactor 

Work Experience in TWo 
Tears Prior to EnrollMnt 
Regular Job lasting 

>3 months 
Other Job lasting 

>3 months 
Other 

School Status at Referral 

Enrolled 
Not enrolled 



0.38 



-1.38 
-1.87 

-1.61 
0.06 



1.12 
0.93 

0.74 
-0.07 



-0.18 
-0.12 



0.51 
-0.84 
1.09 



-0.53 
0.11 



-1.73 
1.54 
2.41 

0.18 
0.06 

-2.07 



1.67 



-3.46 
-3.64 

-2.05 
-1.13 



1.14 
-0.31 

1.48 
0.26 



-0.50 
0.69 



-0.38 
-0,19 
0.78 



-0.70 
0.82 



-0.79 
-0.43 
0.23 

1.84 
1.84 

-1.41 



2.58 



-1.64 
-1.72 

-0.22 
-0.15 



1.28 
1.10 

-1.20 
0.79 



-1.06 
-1.05 



0.16 
-1.67 
0.24 



0.54 
-0.26 



-1.35 
0.16 
-0.90 

-0.24 
0.87 

-1.71 



3.45 



-3.65 
-2.23 

1.07 
0.34 



0.47 
1.38 

-0.72 
-0.84 



-1.36 
-1.01 



-0.48 
-1.87 
-0.34 



0.17 
-0.33 



-1.09 
-0.31 
0.50 

0.90 
2.79 

-0.76 
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TABU A* 3 (continued) 



Outcome Measure and Time Period 



Control Variables 



Earnings In Regular Jobs Whether Any Paid Job 
Month 6 Month 22 Month 6 Month 22 



KKperiMOtAl Status 

Experlaental 

Control 

B*Sitc 

Cincinnati 
Los Angeles 
New York 
St. Paul 
Tucson 

B*IQ UTtl 

Borderline 
Mild 

Moderate 

B*Age 

Younger than 22 
22 or older 



-0.16 



2.41 

2.59 
0.27 



-0.53 
-0.99 

-0.79 



-0.20 



-0.02 
0.57 

0.56 
-0.81 



0.35 
0.91 

-1.25 



0.84 



-0.35 
-0.89 

-0.21 
0.98 



-0.82 
-1.11 

0.74 



-0.90 



1.11 
0.18 

-0.12 
-0.18 



-0.19 
-0.18 

-0.26 



B*Gealer 

Male 

Female -2.11 

B*S«ce/BtlmiclC7 

Black 1.12 

Hispanic 0.42 

White and other — 

B*Ll¥lqg Arraqgeaent 

Living with parents -0.72 
Living In supervised — 
setting 

Living Independently 1.81 

B*Fiiiaiici«l NanageMent Skills 

Independent -0.92 
Not Independent — 

B*X«ceipt of Transfers 

SSI/SSDI -0.54 

Other transfers only -0.55 

No transfers — 

B*Secondary Handicaps 

Secondary handicap — 
No secondary handicap 

B*Cause of Retardation 

Organic -0.91 
Nonorganic 

B*Benef actor 

Benefactor — 
No benefactor 

K*Vork Kzperleoce in Two 
Tears Prior to Enrollaent 

Regular Job lasting 1.00 
>3 months 

Other Job lasting 0.62 
>3 months 

Other — 

E*School Status at Referral 

Enrolled 1.46 
Not enrolled 



-2.54 



0.17 
0.73 



1.20 
0.85 
K48 



1.36 
2.18 



0.65 



0.07 
-1.75 

0.33 



"0.99 



0.72 
1.61 



-0.77 
1.07 
-0.34 



0.30 
0.92 



-0.47 



0.80 
-0.83 

0.40 



-1.16 



0.09 
1.91 



1.10 
0.97 
1.04 



0.39 
1.70 



1.11 



-0.07 
-1.79 

-0.24 



NOTE: All control variables are measured at baseline or referral. The variable definitions and 
their means are reported In Table A. 1. B* Indicates that a control variable has been 
Interacted with Kiq^riaental Status. The corresponding coefficient estimates are reported 
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TABIX A.4 

ESTIMATED PROGRAM IMPACTS ON EARNINGS FOR KEY SUBGROUPS 
OF STETS PARTICIPANTS 

A. EARNINGS FROM REGULAR OOBS* 







Month 6 






Month 15 






Month 22 




Subgroups Defined 
by Ch«r«eterl9tlcs 
«t Baseline 


ExperlMental 
Croup Mean 


Control 
Group 
Mean 


Estimated 
iMpact 


ExperlMental 
Group Mean 


Cnn^ f>n1 

Group 
Mean 


Estimated 
Inpact 


ExperlMental 
Group Mean 


Control 
Group 
Mean 


EstlMated 

iMpact 


Total Sa^U 


11.81 


9.81 


2.00 


26.90 


16.31 


10.59»» 


36.36 


20.55 


15.81** 


Sit* 

Los Angeles 
Mew York 
St. Peul 
Tucson 


17.93 
3.85 

27.58 
8.14 


1.36 
17.99 
-1.06 
18.61 


3.99 
16.57* 

-14.14 
28.64»» 

-10.47 


19. 33 
14.27 
50.44 
27.00 
20.59 


6.67 
-1.47 
42.48 
13.x 
15.94 


12.66 
15.74 

7.96 
13.70 

4.65 


16.79 
27.55 
61.31 
41.90 
32.25 


1.70 
2.42 
45.79 
14.20 
31.28 


15.09 
25.13»* 
15.52 
27.70* 
0.97 


Borderline 

Mild 

Moderete 


13.98 
11.66 
7.42 


13.76 
6.47 

lie 1A 
lo. 10 


0.22 
5.19 
-8.76 


26.90 
24.45 
h/\ 71 


29.69 
10.25 
13.58 


-2.79 
14.20»» 
27. 1 3* 


45.44 
30.68 
43.42 


28.32 
18.09 
13.63 


17.12 

12.59* 

29.79* 


Am 

Younger than 22 
22 or older 


11.17 
13.56 


11.47 
5.29 


-0.30 
8.27 


27.83 
24.83 


20.47 
6.99 


7.36 
17.84* 


35.89 
37.41 


25.51 
9.58 


10.38 
27.83** 


Nele 
Fe««le 


19.80 
6.27 


9.99 
9.57 


9.81» 
-3.30 


32.32 
19.52 


17.13 
15.19 


15.19»» 
4.33 


47.89 
21.39 


19.61 
21.75 


28.28** 

-0.36 


lUoe/EthRlclty 
Bleck 
Mis panic 
White and other 


10. ^ ^ 

12.96 
7.68 


7.U3 

9.15 
10.36 


9.28 
3.81 
-2.68 


16.04 
34.51 
31.38 


13.82 
30.91 
13.87 


2.22 
3.60 
17.51»» 


31.33 
55.85 
34.17 


15.72 
29.67 
20.93 


15.61 

26.18* 

13.24 


Living with parents 
Living In supervised 

setting 
Living Independently 


10.77 
5.83 


12.04 
5.99 


-1.27 
-0.16 


27.11 
10.50 


15.48 
16.54 


11.63»» 
-6.04 


38.17 
19.72 


19.81 
25.67 


18.36** 
-5.95 


31.24 


-10.03 


41.27»» 


43.83 


24.41 


19.42 


38.88 


21.52 


17.36 


Flnaneial Mamgwent Skills 

Independent 


10.62 

1 9 11 


15.04 
/ .0/ 


J^.42 


41.53 
21.38 


14.86 
16.86 


26.67»» 
4.52 


55.98 
28.95 


25.99 
18.49 


29.99** 
10.46 


RtMlpt of Trent fen 

SSI/SSOI 

Other transfers only 
No transfers 


6.36 
11.62 
16.58 


6.68 
11.79 
10.95 


-0.32 
-0.17 
5.63 


22.07 
35.96 
23.17 


16.11 
14.23 
18.44 


5.96 
21.73»» 
4.73 


37.48 
46.92 
24.87 


18.03 
17.63 
25.93 


19.45* 

29.29** 

-1.06 


Cauoe of Retardation 

Organic 
Non-organ Ic 


11.39 


91 1£ 

7.59 


-7. 16 
3.80 


17.17 
28.80 


13.40 
16.88 


3.77 
11.92»» 


40.42 
35.52 


16.24 
21.43 


24. 18* 
14.09** 


Ibric Expert once Iri Too 
Vtars Ptior to EflrollMiit 
Regular job lasting 

>3 Months 
OtTwr job lasting 

>3 Months 
OtKsr 


21.81 
12.52 
7.55 


11.00 
9.85 
9.43 


10.81 
2.67 
-1.88 


43.47 
24.73 
23.59 


27.83 
15.76 
13.35 


15.64 
8.97 
10.24* 


58.37 
29.70 
34.49 


34.47 
27.82 
11.69 


23.90 
1.88 
22.80** 


Softool Status at Referral 

Enrolled 
Not enrolled 


12.64 
11.67 


«0.20 
14.18 


12.84 
-2.51 


22.04 
29.12 


5.42 
21.25 


16.62* 
7.87 


30.06 
39.18 


11.08 
24.79 


18.98* 
14.39** 


Nuober In Saople 






283 






402 






395 



MOTE: These results were estlMated through ordinary least squares techniques. In addition to the control verifies that are Included In the Models 
which underlie the overall net iMpact estlMates reported In Table IV. 4, these Models Include variables that Interacted the treatment varlAle 
with the subgroup variables. The full regression froM which the 6- and 22-Month results were derived are presented In Tables A.2 and A. 3. 

Regular jobs are those that are neither tralnlng/work-study nor workshop/act Ivlty-center jobs. 

*Statlstlcally significant at the 10 percent levels two-tailed test. 
**Statlstlcally significant at the 5 percent level, two- tailed test. 
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TABLE A. 4 



ESTIMATED PROCRAH IMPACTS ON EARNINGS FOR KEY SUBGROUPS 
OF STETS PARTICIPANTS 

6. EARNINGS FROM ALL PAID DOBS 



Subgroups Defined 
by Ch«r«cterist ics 
et Baseline 



Experimental 
Group Mean 



Month 6 



Control 
Group 
Mean 



EstlMated 
Iiipact 



Experimental 
Group Mean 



Month IS 



Control 
Group 
Mean 



Estimated 

Impact 



Experimental 
Croup Mean 



Month 22 



Control 
Group 
Mean 



Estimated 

Impact 



Total Sample 

Sit* 

Cincinnati 
Los Angeles 
New York 
St* Paul 
Tucson 

IQ Uvei 

Borderline 

Mild 

Moderate 



52.19 



M).27 
39.64 
72.32 
51.23 
53.68 



58.59 
51.80 
40.39 



25.93 



15.49 
13.49 
44.78 

22.75 
27.94 



25.51 
24.94 
31.55 



26.46** 



24.78* 

26.15»» 

27.54»» 

28.48* 

25.94»» 



33.08»» 
26.86»» 
8.64 



37.91 



21.36 
24.00 
57.72 
36.01 
46.63 



38.55 
34.86 
53.20 



26.48 



7.96 
7.53 
56.36 
31.49 
27.02 



35.36 
22.06 
26.86 



11.43»» 



13.42 
16.47 
1.36 
4.52 
19.61* 



3.19 
12.60»» 
26.34* 



40.7;^ 



21.54 
28.74 

63.60 
48.60 
40.46 



47.37 
36.33 
47.74 



26.41 



3.74 
6.24 
52.29 
38.48 
36.19 



33.06 
27.50 
21.25 



12.36»» 



17.80 
20.50* 
11.31 
10.12 
2.27 



14.31 
8.83 
26.49 



Younger than 2 2 53.66 25.2 2 28.44»» 39.87 24.60 15.27»» 40.41 31.29 9.12 

22 or older 48.94 27.87 21.07* 33.49 30.68 2.81 41.61 22.02 19.59* 

Qtnter 

Male 57.20 21.59 35.61»» 46.16 29,10 19.06»» 53.5 5 28.00 25.55»» 

Female 46.05 31.67 14.36* 23.90 22.90 1.00 24,23 28.93 ^.70 

lUcm/Ethnicity 

Black 55.27 23.49 31.78»» 32.09 22.79 9.30 34.93 23.87 11.06 

Hispanic 66.36 30.52 37.84»» 53.83 47.52 6.31 62.66 37.47 25.39* 

White and other 47.73 26.41 21.32»» 37.09 23.01 14.08* 38.40 28.64 9.76 

Living Arramgeaent 

Living with parents 50.58 27.89 22.69»» 37.0 5 27.28 9.77* 42.29 27.36 14.91** 

Living in supervised 57.29 26.28 31.01 31.83 25.67 6.16 31.55 35.56 ^.01 
setting 

Living independently 66.14 3.59 62.55»» 53.41 19.36 3C.05* 37.3 5 29.70 7.65 
r'liwrlal NafiagMeiit 5M1 1> 

Independent 55.56 31.96 23.60** 56.79 29.95 26.84»» 60.51 29.o7 30.64 

Not independent 51.09 23.46 27.63»» 30.77 25.17 5.60 33.34 27.93 5.41 

neomlpt of TransDtn 

SSI/SSDI 56.90 25.54 31.36»» 35.35 24.03 11.32 42.93 25.62 17.11 

Other transfers only 50.55 23.80 26.75»» 44.75 23.69 21.06»» 50.43 24.25 26.18»» 

Mo transfers 49.96 27.89 22.09»» 34.01 31.45 2.56 29.19 35.06 -5.89 

Cwee of RotanlatJon 

Organic 47.65 30.77 16.86 36.79 13.20 23.59* 46.07 22.46 23.61* 

Non-organic 5:i.31 24.98 28.33** 36.12 29.05 9.07* 39.70 29.63 10.07* 

Ibtfc Exporionoo In Too 
Years Prior to Enrollment 

Regular Job lasting 64.09 26.63 35.46** 55.49 M).10 15.39 60.55 41.69 18.66 
>3 months 

OtT«r Job lasting 48.3 5 27.69 20.46** 35.99 31.79 4.20 34.72 37.71 -2.99 
>3 months 

OtTJbr 51.82 23.68 28.14** 34.11 18.85 15.26** (2.53) 18.32 -20.85** 
Seh9ol Status at Referral 

Enrolled M.66 14.65 M.21** 26.V6 22.04 4.94 31.69 19.66 12.23 

Mot enrolled 53.93 30.86 23.07** 42.87 28.50 14.37** 44.78 32.33 12.45* 



Number In Sample 



283 



402 



395 



NOTEt These results were estimated through ordinary least squares techniques. In addition to the control variables that are included in the models 
which underlie the overall net impact estimates reported in Table IV.4, these models Include variables that interacted the treatment variable 
with the subgroup variables. 



*Statistically significant at the 10 percent level, two-tailed test. 
**Statistically significant at the 5 percent level, two-tailed test. 
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TABLE A. 5 



COMPARISON OF SELECTED NET IMPACT ESTIMATES BASED 
ON OLS REGRESSION MODELS AND MAXIMUM 
LIKELIHOOD ESTIMATION METHODS 



Outcome Measure 



Reference Period/ 
Estimation Method 



Regular 
Jobs 



Workshop/ 
Training Activity 
Jobs Center 



Any Paid 
Job 



Month 6 
OLS 

Pro bit 



1.1 
-0.2 



27.2** 
30.4** 



-6.3* 
-4.8* 



22.6** 
24.1** 



Month 22 
OLS 

Pro bit 



11.9** 
13.5** 



0.3 
0.1 



-11.3** 
-9,2** 



1.0 
0.1 



NOTE: The OLS estimates are based on the basic regression model specified 
in Appendix Table A.l. The impact estimate based on this model is 
the coefficient on the experimental status variable. The probit 
estimates are based on maximum likelihood estimates of coefficients 
on a similar set of control variables as was used in the OLS 
model. In this case, the experimental effect is estimated as: 



PROB (XB| experimental) - PROB (XB| control) , 
where PROB ( ) is the probability. 



*Statistically significant at the 10 percent level. 
**Statistically significant at the 5 percent levei. 
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TABLE A, 6 



CHOW TEST RESULTS OF THE ACCEPTABILITY OF 
POOLING OBSERVATIONS ACROSS SITES 



Outcome Measure 



F-Statistic 



Degrees of Freedom 



Total Earnings in 
Month 6 



1.27 



86, 192 



Regular Job Earnings 



1.19 



86, 192 



in Month 6 

Total Earnings in 1.06 
Month 22 

Regular Job Earnings 1.10 
in Month 22 



*Statistically significant at the 10 percent level. 
**Statistically significant at the 5 percent level. 
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ESTIMATED COEFFICIENTS IN MODEL*? USED TO GENERATE SELECTED 
NET IMPACT ESTIMATES REPORTED IN TABLE V.4 AND V.5 



Outcome Measure and Time Period 



Control Variables 



Percent In Training 



Month 6 



Percent In School 



Month 22 



Month 6 



Month 22 



Intercept 

Site 
Cincinnati 
Los Angeles 
Net# York 
St. Paul 
Tucson 

IQ Lerel 

Borderline 
Mild 

Moderate 

Younger than 22 
22 or older 

Gender 
Mal^ 
Female 

Sace/Ethalclty 

Black 

Hlsiwnic 

White and other 

LlvlQg Arraiweaent 

Living with parents 
Living In supervised 

setting 
Living Independently 

Flnudal Nanagesent Skills 

Independent 
Not Independent 

Receipt of Transfers 

SSI/SSDI 

Other transfers only 
No transfers 

Secondary Handlers 

Secondary handicap 
No secondary handicap 

Canoe of Retardation 

Organic 
Non~organlc 

Benefactor 

Benefactor 
No benefactor 

Vork Bzp^rionce In Ttio 
Tears Prior to Enrollaent 

Regular Job lasting 

>3 months 
Otlier job lasting 

>3 months 
OtTier 

School Status at Referral 

Enrolled 
Not enrolled 

Experimental Status 

Experimental 
Control 



82.6 



-22.2 
-7.1 

-2.9 
-5.0 



-1.3 
6.4 

-21.4 



1.0 



-8.1 
-12.7 



-25cj 
-51.6 
5.7 

l6i9 
2.1 
-7.3 
-16.8 

5.2 
13.9 

-9.2 

2.0 



54.5 



-2.6 
0.1 

6.5 
1.6 



-11.3 
-9.1 

-21.1 
0.4 



-12.4 
-19.0 



-0.6 
-23.0 
-9.2 



6.8 
6.3 



-6.8 
23.3 
0.7 

-3.5 

-4.7 

-19.4 



-7.2 



-3.1 
14.6 

4.8 
-0.1 



-7.3 
-6.8 

-1.5 



1.0 



0.6 
0.8 



22.1 
-0.3 
15.2 



-3.2 
-5.5 



-7.5 
8.7 
1.9 

-3.6 
2.7 

12.1 
15.0 



-11.2 



-4.9 
13.3 

-6.7 
3.9 



-0.6 
12.0 

13.2 



1.2 



-12.0 
-17.2 



15.4 
5.1 
-0.8 



8.8 
0.4 



3.1 

-5.2 
-2.2 

-10.4 
-3.4 

10.9 
6.9 
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TABLE A. 7 (continued) 



Outcome Measure and Time Period 



Percent in Training 



Percent in School 



Control Variables 



Month 6 



Month 22 



Month 6 



Month 22 



B*Slte 

Cinci nnati 
Lo3 Angeles 
Kcw York 
St« Paul 
Tucson 

B*IQ Level 

Borderline 
Mild 

Moderate 

S*A|{e 

Younger than 22 
22 or older 

B*G€nder 
Male 
Female 

K*Race/Ktlmlcity 

Black 

Hispanic 

White and other 

B*LlTiqK Arraqgeaeat 

Living vlth parents 
Living in supervised 

setting 
Living independently 



B*Fi]iai»clal Manage 

Independent 
Not independent 



Bat Skills 



K*Eecelpt of Tramfen 

SSI/SSDI 

Other transfers only 
No transfers 

S*SecoQdaz7 Handicaps 

Secondary handicap 
No secondary handicap 

B*C«ase of Ketardatlon 

Organic 
Non-o rganic 

K*Benef actor 

Benefactor 
No benefactor 

K*Vork Experience In Two 
Teara Prior to Earollaent 

Regular Job lasting 

>3 nonths 
OtlTer Job lasting 

>3 months 
Ot¥er 

K*Scliool Status at Referral 

Enrolled 
Not enrolled 



-3.6 
-35.2 

-33.0 
12.7 



-9.8 
-1.5 

23.1 
10.1 



4.4 
20.4 



7.7 
17.8 
2.5 



-6.4 
7.5 



10.6 



-3.7 
-9.4 



11.8 



1.1 
2.7 

7.7 



9.4 
14.4 

16.6 
-4.4 



4.5 
10.8 



-8.5 
-0.5 
2.3 



0.4 
-10.8 



-15.8 



-8.5 
-6.0 



-4.4 



-1.8 
•13.8 

-5.5 
M3.3 



15.5 
11.4 

1.6 
-2.9 



2.3 
14.9 



-17.9 
14.3 
-18.5 



-5.7 
7.1 



-7.5 



-0.1 
-9.8 

-12.5 



7.5 
-1.1 

6.3 
1.9 



-4.7 
-21.0 

-10.3 
4.0 



14.9 
13.6 



-11.8 
-11.5 
2.7 



-2.8 
1.7 



7.0 



4.1 
5.0 



-11.8 



NOTE: All control variables are measured at baseline or referral. The variable definitions and 
their means are reported in Table A. 1. B* indicates that a control variable has been 
interacted with Kxperlmental Status. The t-statistics for the coefficient estimates are 
reported in Table A. 8. 
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TABLE A. 8 



t-STATISTICS FOR ESTIMATED COEFFICIENTS IN MODElS USED TO GENERATE 
SELECTED NET IMPACT ESTIMATES REPORTED IN TABLE V.4 AND V.5 



Outcome Measure and Time Period 



Control Variables 



Percent in Training 



Percent in School 



Month 6 



Month 22 



Month 6 



Month 22 



Intercept 

Site 

Cincinnati 
Los Angeles 
New York 
St. Paul 
Tucson 

IQ Level 
Borderline 
Mild 

Moderate 
Age 

Younger than 22 
22 or older 

Gender 

Male 
Female 

Eace/Etfanlclty 

Black 

Hispanic 

White and other 

LlTlqg Arraqgeaent 

Living with parents 
Living in supervised 

setting 
Living independently 

Financial Manageaent Skills 
Independent 
Not independent 

Receipt of Transfers 

SSI/SSDI 

Other transfers only 
No transfers 

Secondary Handicaps 

Secondary handicap 
No secondary handicap 

Cause of tetazdation 

Organic 
Non-organic 

Benefactor 

Benefactor 
No benefactor 

Work Experience in Two 
Tears Prior to Enrollaent 

Regular Job lasting 

>3 months 
Otlier Job lasting 

>3 months 
OtTTer 

School Statua at Referral 

Enrolled 
Not enrolled 

Ezperlaental Status 

Experimental 
Control 



3.12 



-1.75 
-0.56 

-0.17 
-0.36 



-0.14 
0.44 

-1.81 
0.11 



-0.76 
-0.86 



-1.51 
-1.92 
0.60 



1.78 
l.OI 



0.34 
-0.59 
-2.32 

0.41 
1.53 

-0.93 
0.06 



3.17 



-0.27 
0.02 

0.55 
0.16 



-1.66 
-0.83 

-2.58 



-0.06 



-1.57 
-1.97 



-0.05 
-1.39 
-J. 26 



0.80 
0.82 



-1.51 
2.56 

0.13 

-0.37 
-0.70 

0.02 
-0.88 



-0.39 



-0.35 
1.65 

0.41 
-0.02 



-1.12 
-0.65 

-0.19 
0. 16 



0.09 
0.08 



1.85 
-0.02 
2.26 



-0.40 
-0.73 



-1.69 
0.99 
0.68 

-0.41 
0.42 

1.74 
0.65 



-0.91 



-0.71 
2.04 

-0.79 
0.55 



-0.12 
1.53 

2.24 
0.27 



-2.12 
-2.47 



1.81 
0.43 
-0.16 



1.44 
0.07 



0.96 
-0.80 
-0.59 

-1.54 
-0.70 

2.05 
0.43 



TABLE A. 8 (continued) 



Control Variables 



Outcome Measure and Time Period 



Percent in Training 
Month 6 Month 22 



Percent in School 



Month 6 



Month 22 



K*51te 

Cincinnati -0.19 

Los Angeles -1.95 

New York — 

St. Paul -1.42 

Tucson 0.66 

K*IQ Lev«l 

Borderline -0.74 
Mild 

Moderate -0.08 

Younger than 22 1.50 
22 or older 

B*Geiider 

Male — 
Fenale 0.83 

K*Eace/Btfaalclty 

Black 0.30 

Hispanic 0.97 

White and other — 

K*LlTiqK Arraqgewnt 

Living with parents 0.35 

Living in supervised — 
setting 

LlviQg independently 0.55 

K*Pliiaiicl«l Managment Skills 

Independent 0.18 
Not independent ->~ 

K*lecelpt of TrsQsfers 

SSI/SSDI -0.41 
Other transfers only 0.49 
No transfers — 

K*Secondsr3r Handicaps 

Secondary handicap — 
No secondary handicap 

E*C«asa of tetardation 

Organic -0.62 
Non-organic — 

B*Beiiaf actor 

Benefactor — 
No benefactor — 

K*Work Kxporlence In Two 
Teats Prior to Snrollaent 

Regular Job lasting -0.21 
>3 nonths 

OtlTer Job lasting -0.74 
>3 aonths 

OtTTer — 

K*School Status at Kef erral 

Enrolled 0.82 
Not Enrolled ~ 



0.08 
0.21 

0.30 
0.57 



0.97 
1.00 

1.57 



-0.52 



0.42 
0.81 



-0.56 
-0.03 
0.23 



0.04 
-1.03 



-1.34 



-0.67 
0.66 

-0.42 



-0.14 
-0.19 

-0.34 
-0.02 



1.65 
0.81 

0.15 
-0.34 



0.22 
1.00 



-1.17 
0.63 
-1.88 



-0.51 
0.67 



-0.62 



-0.01 
-1.09 

-1.23 



0.78 
-0.12 

0.53 
0.20 



-0.67 
-2.03 

-1.34 
0.65 



1.91 
1.41 



-1.07 
-0.77 
0.38 



-0.34 
0.23 



0.83 



0.44 
0.77 

-1.56 



NOTE: All control variables are measured at baseline or referral. The variable definitions and 
their aeans are reported in Table A.l. K* indicates that a control variable has been 
Interacted with Experimental Status. The t-statistics for the coefficient estimates are 
reported in Table A. 7. 
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TABLE A. 9 



ESTIMATED COEFFICIENTS IN MODELS USED TO GENERATE SELECTED 
NET IMPACT ESTIMATES REPORTED IN TABLE VI. 3 AND VI. 4 



Outcoae Measure and Time Period 



Control Variables 



Monthly Income from 
SSI/SSDI 



Monthly Income from 
Other Cash Transfers 



Month 6 



Month 22 



Month 6 



Month 22 



Intercept 


-30.3 


111.4 


52.6 


12.3 


Site 

Cincinnati 
Los Angeles 
New York 
St. Paul 
Tucson 


15.4 
69.8 

-8.3 
42.5 


-21.0 
81.3 

-95.5 
-29.2 


-15.2 
-36.6 

31.3 
-33.8 


16.4 
-3.7 

29.4 
11.0 


IQ LeTeX 

Borderline 
Mild 

Moderate 


0.5 

20.0 


-60.4 
-58.6 


-11.7 
"7.3 


3.0 
-14.0 


Age 

Younger than 22 
LL or oiaer 


25.2 


-25.8 


-22.3 


-23.4 


Gender 
Male 
Female 


7 1 
/•I 


-31.7 


-12.5 


12.6 


tacc/Btfanlcity 

Black 

Hispanic 

wnlte and other 


-24.7 
-22.6 


-49.2 
-47.1 


-11.1 
-9.9 


24.5 
10.4 


LlTlqg Arrai^eaent 

Living with |>arents 
Liivxiig m Bupervisea 

setting 
Living independently 


-3.4 
-5.9 


8.3 
126.4 


-25.9 
-50.5 


-22.4 
-41.2 


V4MMnM4 a1 ** — 6WX 1 1^ 

Fxnenci fianageaenc skxIXb 

Independent 
Not independent 


7.9 


3.3 


27.0 


31.8 


Receipt of Transfers 

SSI/SSDI 

Other transfers only 
No transfers 


180.1 
3.7 


116.9 
50.5 


14.7 
63.5 


6.1 
29.6 


Secondary Bsndicaps 

Secondary handicap 
No secondary handicap 


10.5 


32.7 


8.5 


11.0 


Cause of Retardation 

Organic 
Non-*organic 


-50.8 


80.1 


2.8 


-11.2 


Benefactor 

Benefactor 
No benefactor 


18.0 


7 1 


6.0 


-15.0 


Vork Ixperience in Two 
Tears Prior to Enrollment 

Regular Job lasting 

>3 nonths 
OtTTei: job lasting 

>3 months 
OtTfer 


8.4 
16.8 


5.7 
-47.8 


-3.2 
-1.0 


22.7 
-9.4 


School Status at Referral 

Enrolled 
Not enrolled 


-3.4 


17.5 


15.1 


14.4 


Bxperiaental Status 

Experimental 


6.8 


-92.4 


-65.5 


-38.3 
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TABLB A. 9 (continued) 



Control Variables 



Outcome Measure and Time Period 



Monthly Income from 
SSI/SSDI 



Month 6 



Month 22 



Monthly Income from 
Other Cash Transfers 



Month 6 



Month 22 



K*Slte 

Cincinnati -16.8 

Los Angeles -22.7 

New York — 

St. Paul -38.7 

Tucson -25.9 

K*IQ Ural 

Borderline -6.4 
Mild 

Moderate 11.8 
K*Age 

Younger than 22 -44.2 
22 or older 

B*Geiider 

Hale 

Female -21.8 

K*K«ce/Btbnlclty 

Black -19.0 

Hispanic -14.4 

White and other — 

K*LlYlqg Arraqgeaeat 

Living with parents 41.9 

Living In supervised — 
setting 

Living Independently 115.9 

B*Plnaiiclal Hanageaent Skills 

Independent -8.4 
Not Independent — 

E*Eecelpt of Transfers 

SSI/SSDI -13.0 

Other transfers only 9.4 

No transfers — 

B^Secondary Handicaps 

Secondary handicap 
No secondary handicap 

E^Canse of Retardation 

Organic 79.5 
Non-organic 

E*Beiief actor 

Benefactor — 
No benefactor — 

E*Vozk Ezperlei^c^ in TWo 
Tears Prior to Enrolljaeat 

Regular Job lasting -10.5 
>3 months 

OtlTer Job lasting -1.5 
>3 months 

OtTTer — 

E*School Status at Referral 
Enrolled 11.1 
Not enrolled 



3.0 
13.9 

59.6 
37.5 



52.0 
92.4 

-17.5 



49.4 



-1.6 
23.4 



15.7 
-107.7 
-12.3 



50.0 
-32.7 



-87.7 



-4.9 
59.0 

7.7 



-1.6 
17.2 

21.9 
7.8 



7.8 
19.4 

34.4 
24.4 



0.9 
7.9 



28.7 
48.1 
-25.4 



1.5 
-32.9 



0.1 



-12.7 
10.0 

-19.2 



-2.2 
-2.8 

7.3 
-23.8 



-2.0 
26.1 

39.8 
9.4 



6.6 
-18.1 



29.3 
57.6 
-16.6 



-3.8 
-25.9 



5.2 



-32.2 
34.1 

-6.8 



NOTE: All control variables are -measured at baseline or referral. The variable definitions and 
their means are reported In Table A. 1. B* Indicates that a control variable has been 
interacted with Experlaeatal Status. The t-statlstlcs for the coefflclenl estimates are 
reported In Table A. 10. 
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TABLE A. 10 



t-STATISTICS FOR ESTIMATED COEFFICIENTS IN MOMLS USED TO GENERATE 
SaECTED NET IMPACT ESTIMATES REPORTED IN TABLE VI. 3 AND VI. 4 



Outcome Measure and Time Period 



Control Variables 



Monthly Income from 
SSI/SSDI 



Month 6 



Monthly Income from 
Other Cash Transfers 



Month 22 



Month 6 



Month 22 



Intercept 

Site 

Cincinnati 
Los Angeles 
New York 
St. Paul 
Tucson 

IQ Level 

Borderline 
Mild 

Moderate 
Age 

Younger than 22 
22 or older 

Gender 

Male 
Female 

Race/EthnlclCy 

Black 

Hispanic 

Whit3 and other 

UviOg Arraogeaent 

Living with parents 
Living in supervised 

setting 
Living independently 

Financial Hanageaent Skills 
Independent 
Not independent 

Kecelpt of Transfers 

SSI/SSDI 

Other transfers only 
No transfers 

Secondary Handicaps 

Secondary handicap 
No secondary handicap 

Cause of Retardation 

Organic 
Non-organic 

Benefactor 

Benefactor 
No benefactor 

Voxk Experience In Two 
Tears Prxo& tA# rTarolljMnt 
Regular Job lasting 

>3 months 
OtlTer Job lasting 

>3 months 
OtTTer 

School Status at Referral 
Enrolled 
Not enrolled 

Experlaental Status 

Experimental 
Control 



-0.54 



0.58 
2.66 

-0.24 
1.47 



0.02 
0.64 

1.02 



0.40 



-1.13 
-0.75 



-0.09 
-0.10 
0.39 



7.66 
0.17 



0.79 
-1.97 
1.19 

0.31 
0.88 

-0.17 
0.10 



1.99 



-0.67 
2.72 

-2.54 
-0.92 



-2.74 
-1.67 

-0.97 



-1.59 



-1.93 
-1.51 



0.22 
2.31 
0.14 



4.30 
2.05 



2.25 
2.75 
0.42 

0.19 
-2.22 

0.73 
-1.28 



1.46 



-0.88 
-2.16 

1.39 
-1.82 



-0.94 
-0.37 

-1.40 
-1.08 



-0.79 
-0.51 



-1.09 
-1.38 
2.04 



0.97 

4,39 



1.00 
0.17 
0.62 

-0.18 
-0.08 

1.14 
-1.46 



0.40 



0.95 
-0.23 

1.40 
0.63 



0.25 
-0.64 

-1.62 



1.15 



1.75 
0.60 



•1.08 
-1.40 
2.42 



0.41 
2.17 



1.38 
-0.20 
-1.62 

1.36 
-0.79 

1.09 
-0.97 
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TABLE A. 10 (continued) 



Outcome Measure and Time Period 

Monthly Income from 



Control Variables 



Month 6 



SSI/SSDI 



Month 22 



Monthly Income from 
Other Cash Transfers 



Month 6 



Month 22 



E*Site 

Cincinnati 
Los Angeles 
New York 
St. Paul 
Tucson 

B*IQ UTel 

Borderline 
Mild 

Moderate 

E*Age 

Younger than 22 
22 or older 

E*Gend6r 
Male 
Female 

E*X«ce/Btbnicity 

Black 

Hispanic 

White and other 

E*LlTiqg Arraqgeaent 

Living with parents 
Living in supervised 

setting 
Living independently 



E*Finaiiclal Managei 

Independent 
Not independent 



tut Skills 



E*Receipt of Transfers 

SSI/SSDI 

Other transfers oaly 
No transfers 

K*Secoiidar7 Handicaps 

Secondary handicap 
No secondary handicap 

E*Cause of Retardation 

Organic 
Non-organic 

E*Beiief actor 

Benefactor 
No benefactor 

E*Vork Experience in Two 
Tears Prior to Enrollment 

Regular Job lasting 

>3 months 
OtlTer Job lasting 

>3 months 
OtTTer 

E*School Status at Referral 
Enrolled 
Not enrolled 



-0.44 
-0.60 

-0.79 
-0^64 



-0.23 
0.28 

-1.36 



-0.86 



-0.61 
-0.33 



0.91 
1.69 
-0.27 



-0.40 
0.30 



2.18 



-0.28 
-0.06 

0.37 



0.07 
0.33 

1.13 
0.85 



1.66 
1.97 

-0.51 



1.79 



-0.05 
0.54 



0.31 
-1.57 
-0.38 



1.36 
-0.96 



-2.30 



-0.12 
2.01 

0.23 



-0.07 
0.71 

0.69 
0.30 



0.44 
0.72 

1.66 
1.50 



0.04 
0.28 



0.95 
1.09 
-1.34 



0.07 
-1.62 



0.00 



-0.52 
0.59 

-0.99 



-0.09 
-0.12 

0.25 
0.98 



-0.12 
1.03 

2.11 
0.62 



0.35 
-0.76 



1.08 
1.55 
-0.93 



-0.19 
-1.37 



0.25 



-1.40 
2.11 

-0.36 



NOTE: All control variables are measured at baseline or referral. The variable definitions and their 
means are reported in Table A. 1. E* indicates that a control variable has been interacted with 
Exp«rimeDtal Status. The t-statistics for the coefficient estimates are reported in Table A.9. 
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TABUE A.11 



/«EA UNDfLOYMENT RATES DURING THE DEHONSlRAnON 
AMD EVALUAHON PERIODS 
(Reroent of the Laborforoe) 



Slte^ 



United States" 



1961 



April 

May 

Oune 

3ijly 

August 

Septenter 

October 

Novwnber 

Oecenber 



Oanuary 

February 

March 

April 

Hay 

CKfie 

July 

August 

September 

October 

November 

Deoenter 



Oaaiary 

February 

March 

April 

Hay 

3ine 

July 

August 

Scptenber 

October 

Novenber 

December 



rm 



Oanuary 

February 

March 

April 

Hay 

CkDe 

Ouly 

August 

Scptentoer 



Cincinnati 



Los 
Angeles 



new 
York 



St. 

Paul 



Tucson 



7.3 
7.3 
7.8 
8.8 
SA 
9.2 
9.1 
9.8 
10.0 



10.7 
10.9 
11.0 
10.2 
9.6 
10.9 
10.6 
11.1 
10.2 
11.2 
11.1 
11.1 



9.3 
8.8 
8.7 
8.5 
8.6 
8.2 
7.9 
8.3 
8.3 



6.9 
6.2 
6.6 
7.0 
6.5 
7.3 
7.5 
6.8 
7.5 



8.9 
8.3 
8.6 
8.5 
BA 
8.8 

9.1 

9A 
10.2 
10.5 

^0A 



BA 
7.5 
7.9 
7.3 
7.1 
SA 
9.5 
8.7 
8.0 



13.7 
7e9 
7.6 
8.8 
7.5 
BA 
BA 
7.7 
8.3 



8.6 
8.6 
9.2 
8.3 
8.1 
9.1 
8.9 
9.7 
8.6 
9A 
8.8 
8.5 



3.9 
6.9 



5.8 
5.9 
6.0 
5.6 
5.7 
6A 
6.3 
6A 
6.5 
6.8 
7.5 
6.9 



^^.9 
^^.6 
5.3 
5A 
5.2 
^.9 
5A 
5.9 
5.7 



7.5 
7A 
7.8 
7.6 
BA 
9.9 
10.3 
^0A 
10.2 
10.3 
10.2 
10.3 



7.6 
8.5 
7.8 
7.1 
7.0 
9.0 
10.3 
9.2 
8.0 



5.5 
5.3 
5.0 
^.7 
^.5 
5.0 
5.0 
^.5 
^A 



5.8 
5.3 
^.9 
^A 
^A 
^.3 
«f.O 
^.0 
3.6 



Total 



7.3 
7.6 
7.3 
7.0 
7.2 
7.5 
8.0 
BA 
8.6 



8.6 
8.8 
9.0 
9.3 
9A 
9.5 
9.8 
9.9 
10.2 
10.5 
10.7 
10.8 



8.0 
7.8 
7.8 
7.8 
7.5 
7.1 
7.5 
7.5 
7A 



20 to 2^f 
Years Olds 



12.0 
12.6 
12.1 
11.5 
12.1 
12.3 
12.7 
.3.0 
13.5 



13.5 

n.^ 
n.5 
n.5 
n.5 
n.7 

15.3 
15.3 
15.8 
16.3 
16.0 



11.5 


11.0 


9.0 


8.1 


11.2 


VA 


16.1 


12.2 


11.5 


8.7 


8.0 


11.6 


^0A 


16.1 


11.9 


10.1 


10.1 


7.9 


11.3 


10.3 


^5A 


10.7 


9.7 


8.6 


7.2 


^0A 


10.2 


^5A 


10.7 


10.1 


8.3 


6.6 


9.8 


10.1 


15.5 


10.8 


10.2 


8.5 


6.9 


9.9 


10.0 


n.5 


9.3 


10.5 


9.7 


6.5 


9.5 


9.5 


13.9 


9.1 


10.8 


9.6 


6A 


8.6 


9.5 


nA 


9.7 


9A 


9A 


5.6 


7.5 


9.3 


13.8 


9.0 


8.7 


BA 


5A 


7.3 


8.8 


13.6 


9.5 


7.8 


7.5 


5A 


6.9 


BA 


13.0 


9.2 


7.0 


7.3 


5.7 


6A 


8.2 


12.0 



12.5 
11.6 
11.6 
12.2 
11.5 
10.7 
11.3 
11.8 
11.5 



These data are from Labor Foroe and Unemplo yment by State and Selected Metropolitan Areas . Bureau of Labor 
adjusted^' w>v«rnmen\: Printing Off Ice, {various Issues.V Ihe data are not seasonally 

b 

These data are seasonally adjusted figures from Employment and Earnings. Bureaj of Labor Statistics. 
Washington, D.C.: U.S. Government Printing Offi ce^ (various issues.) 
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TABLE A. 12 



NUHflCR or PARTICIPANTS AND CONTROLS WITH VARIOUS CmRAaERISTICS 
USED TO OEFirC KEY SUBGROUPS 



Characteristics 



Month 6 



Hoftth IS 



Month 22 



at Baseline 


Experiment als 


Controls 


Experiaentals 


Controls 


Experiment als 


Controls 


Total 


H5 


136 


204 


198 


ax 


169 


Site 














Cincinnati 


29 


29 


39 


44 


44 


34 


Los Angeles 


30 


27 


47 


39 


46 


36 


New York 


35 


34 


49 


47 


44 


50 


St. Paul 


16 


16 


24 


26 


c6 


25 


Tucson 


33 


30 


45 


42 


46 


42 


IQ Level 














Borderline 


41 


42 


57 


61 


54 


60 


Mild 


84 


61 


123 


116 


126 


111 


Moderate 


20 


15 


24 


19 


26 


16 


Age 














Younger than 22 


100 


107 


134 


144 


135 


137 


22 Q~ elder 


45 


31 


70 


54 


71 


52 



Gender 
Male 
Female 



Race/Ethnlelty 
Black 
Hispanic 
White and other 



41 
14 

90 



49 
17 

72 



61 
28 
115 



68 
30 
100 



59 
27 
120 



64 

29 
96 



Living Arranffement 

Living with parents 
Living in supervised 

setting 
Living Independently 



113 
15 



17 



122 
6 



10 



160 
16 

22 



171 
16 

11 



161 

23 



22 



161 
17 

11 



Finmial 

Independent 
Not independent 



SldXls 



42 
103 



40 
96 



61 

143 



49 
149 



61 
145 



47 
142 



Receipt of Transfers 

SSI/SSDI 

Other transfers only 
No transfers 



47 
44 
54 



45 

39 
54 



66 
61 
75 



67 
66 
63 



67 
61 
78 



66 
69 
54 



Cause oC Retardation 

Organic 
Non -organic 

Woric Experience In '*^wo 
Years Prior to Enrollment 

Regular job lartlng 

>,3 months 
Other job lasting 

^3 months 
Other 



23 
122 



23 
52 
70 



23 
115 



19 
50 
69 



37 
167 



31 
74 
99 



28 
170 



16 
66 
114 



38 
168 



31 
71 



29 
160 



27 
64 
98 



School Status at Referral 

Enrolled 
Not enrolled 



36 
109 



50 
88 



53 
151 



72 
126 



52 
154 



70 
119 



Total in Sample 



283 



402 



395 



NOTE: These figures pertain to the samples used to estimate the employment results reported in Table IV* 6« The sample sizes 
differed slightly for other subgroup analyses, due to different patterns of missing d^ta for the outcome measure* 



A. 20 



264 



ESTimTH) PROGRAM WAaS CN T>C PERCENT IN WRKSGPS OR ACHVITY CENTERS, 
BY KEY SUeCnOUPS OF STETS P/^nOP/WTS 







U lit 1 c 

PDntn 6 






Month 15 






Month 22 




Suhonuaa DeflfMl 




Control 






Control 






Control 




by Chirictoristics 


Experljiental 


(k^jup 


EstlABted 


experimental 


Croup 


EstlMted 


Experimental 


Group 


EstlMted 


Jt Baseline 


QrauD Mean 


Ne«) 


Ii«>act 


Qraup Mean 


Me«t 


Impact 


Qroup Mean 


Mean 


Impact 




7,8 


14.5 


-6.7* 


6.4 


15.6 


-9,2»» 


7,9 


19,6 


-11,7«« 


Site 




















Clndnatti 


56.9 


4.4 


13.2 


-2.0 


10.0 


-12.0* 


5,5 


13,6 


-8,1 


Los Angeles 


-2,1 


22.6 


-24.7** 


9.2 


12.6 


-3.4 


4,1 


12,8 


-8,7 


ntfH TOTK 




16 .9 


-14. 6* 


2.6 


18.0 


-15.4»» 


6,2 


18,6 


-12,4« 


St. P«il 


23.1 


5.4 


17.7 


17.3 


25,0 


-7,7 


12.9 


M,1 


-35,2»» 


Tucson 


5.4 


18.8 


-13.4 


9.4 


15.8 


-6,4 


12.2 


15,5 


-3,3 






















Donicrxinc 


Q 9 

y.c 


11 1 

11.1 


_9 V 


7.% 


4K 4 


-4.8 


9,2 


10,9 


-1.7 


HUd 


13.0. 


15.2 


-\6 


4.3 


12.8 


-8.5^ 


6,2 


23,1 


-16.9 


Moderate 


CO*' 


21.8 


-il.8« 


10.3 


35.3 


-25.0^ 


9,3 


22,9 


-13,6 






















•ounger uion c£ 


6.* 


11.1 




5.7 


11.4 


-5.7 


7,0 


13,9 


-6,9* 


or oioer 


13./ 


23.8 


-10.1 


8.2 


25.2 


-17. 0»* 


9,7 


32,0 


-22,3»» 


Gender 




















fuie 


0.1 


14.4 


-6.3 


9.0 


17.7 


-8.7»« 


9,8 


22^4 


-12,6»« 




7 


IK C 




Z.6 


12.7 


-9.9** 


5,3 


15,9 


-10,6»* 


Raoe/Etlwdcily 






















/.* 


13.4 


-6.0 


12.1 


16.2 


-4,1 


9.2 


17.8 


-8,6 


nispanic 


17.6 


2.8 


16.8 


2.9 


15.6 


-12,7 


8,4 


12,1 


-3,7 


mite and ccner 


5.7 


17.4 


-11. 7** 


4.0 


15.3 


-11,3»» 


7,0 


22,5 


-15,5«» 






















Living with parents 


8.7 


16.1 


-7.4* 


6.2 


16.0 


-9,8»» 


7,9 


21,0 


-13,1»* 


Living In supervised 


7.6 


19.3 


-9.7 


14.5 


24.7 


-10,2 


13.8 


24,6 


-10,8 


setting 


















Living Independently 


5.^ 


0.0* 


5.3 


4.8 


0.8 


-1.5 


•0.6 


0,0* 


0,6 


rlMnaai NM^Hnt Skills 




















Independent 


6»7 


8.7 


-1.8 


1.6 


11.3 


-9,7 


2,5 


13,5 


-11,0* 


noc inaepenoent 


O 9 


16.8 


-8.6* 


8.2 


17.2 


-9,0»» 


9,8 


21,8 


-12,0«» 


wwyc or Transrus 




















SSI/SSDI 


16.2 


15.1 


3.1 


12.1 


16.1 


-4,0 


12,1 


22,8 


-10, 7» 


Other transfers only 


1.6 


19.4 


-17.8*H» 


2.8 


15.7 


-12,9<H» 


7,3 


14,5 


-7,2 


Ko transfers 


3.4 


10.1 


-6,7 


4,4 


15.1 


-10,7»» 


4,0 


21,1 


-17,1»« 


Cajoe of Wat Kdlfciii 




















Organic 


8.4 


28.4 


-20.0»» 


10.3 


27.9 


-17,6»» 


16,5 


23,3 


-6,8 


Mcxi -organic 


7.6 


11.7 


-4.1 


5,6 


13,1 


-7,5»» 


6,1 


18,8 


-12,7*» 


lacfc ExpMlenoB In Jm 




















Years mor to Enrol laertt 














0.0^ 






Regular )ob lasting 


-0.4 


10.0 


-10.4 


0.9 


15.7 


-14,7» 


5,4 


-5,4 


>3 iK>nth5 


















OtT^r job lasting 


7.5 


14.4 


-6.9 


9.5 


21.5 


-12,0w 


11,6 


26,4 


-14,8« 


>3 months 




















Otittr 


10.3 


15.9 


-5.5 


5.9 


11.7 


-5.8 


7.8 


19,2 


-11,4*« 


SdMMl SUbis 4rt Referrel 




















Enrolled 


1.3 


12.8 


-11.5 


3.3 


20.8 


-17,5»» 


4,1 


16,7 


.12,6»» 


Not enrolled 


10.6 


15.2 


-4.6 


7.9 


13.3 


-5,4 


9,6 


20,9 


-11,3»«' 



Hmber In Sample 287 414 402 

NOTEt These results were estimated throu^ ordinary least squares techniques. In addition to the control variables that are Included In the 
models which ifvlerlle the (^rall net lji|>act estimates nport In TAle IV,4, these models Include varlAles that Interacted tte 
treatment variable with the subgroup variables, 

a 

The control ^oup mean value was actually calculated to be sll^Uy negative because oT the li^preclslon oT OLS estimation with a bln«7 
outcome varlible. 
b 

The experimental ^roup mean value was actually calculated to be sll^tly negative because of the li^preclslon of OLS estimation «lth a bln«7 
outcome varlAle, 

"Statistically significant at the 10 percent level, two-tailed test. 
*^atlstlcally significant at the 5 percent level, two tailed test. 
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OISTRIBUnCW OF TOTAL ICEXLY INCO^C 



Month 6 Month 15 Month 22 

Experlireneal Cxperlmntal Cxperlfiertal 
Experimental Control Control Cxperlmetital Control Cbntrol Experimental Control Control 
Ooup Qroup Difference Ctov Qroup Difference C^oup C^oip Difference 

AwoMtagi DlstriiMtion 



50 19.01 27.07 -6.06 18.78 20.00 -1.22 20.S9 19.$8 1.01 

$1-S60 17.61 39.10 -21.49 32.99 36.<^1 -3.42 26,47 32.28 -5.81 

$61-$100 37.32 U.29 23.03 16.75 22.05 -5.30 21.57 3.0^ ^.47 

H>re than $100 26.06 19.55 6.51 31.47 21.54 9.93 31.37 20.11 11.26 

Avenge $73.78 $^9.63 $24.15 $70.62 $58.24 $12.38 $72.52 $60.69 11.83 

Hartian $80.21 $35.00 $45.21 $S9.51 $50.46 $9.05 $62.80 $55.76 7.0<^ 



tOnber in Sanple W 133 275 197 195 392 204 169 393 



NOTE: These data are not regression-adjuf^'sd. Ttus, the nean values (averages) differ slightly froM the figures reported in Table VII. 5. 
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TABLE A. 15 



AVERAGE OPERAHNG EXPENDITURES FOR TRANSITIONAL AND 
SUPPORTED EMPLOY^CNT PROGRAMS 
(1982 dollars) 



Program 



Average 
Expenditure 
per Client 
Year 



Alternative Prograns for Nentolly 
Retarded Young Adults 

Special Education, EHR 

(Kakalik et al., 1961) 
Special Education, THR 

(Kakalik et al., 1961) 
Sheltered Workshops 

(U.S. Department of Labor, 1977) 
Work Activity Centers 

(U.S. Department of Labor, 1977) 

STETS Program (RicdLo ml Price, 198^) 

Cincinnati 
Los Angeles 
New York 
St. Paul 
Tucson 

Average for all participants 

Virginia Canonvealth Uhiversityt 
Rehabilitation Research and Training 
CMter 

Transitional Employment (Hill 

et al., 1963b) 
Supported Employment (Hill 

et al., 1965b) 
Project Employ ability (Hill 

at al., 1965a) 

Bay State Skills.Coiporation 
(Bailis, 198^)^ 

University of Washington Food 
Service Prograa (Moss, 1980) 



5,617 
6,166 
5,920 
2,525 



10,311 
6,7^^3 

11,^^67 
5,^^11 

6,715 



7,119 
2,m 
n.a. 

9,260 

10,771 



Average 
Expenditure 

per 
Participant 



n.a. 
n.a. 
n.a. 
n.a. 



6,420 
7,266 
9,651 
4,263 
7,060 
7,553 



3,266 
n.a. 
6,264^ 

3,260 

9,560 



NOTE: 



While an attempt was made to make the cost estimates as consistent as possible, 
differences still exist. For example, the costs for STETS and the Bay State Skills 
Corporation include some wage payments to participants, while the Virginia Commonwealth 
estimates do not include any payments to clients. Costs have been inflated to 1984 
dollars by using the change in the implicit price deflator for gross national product. 

n.a. means that data are unavailable. 

^Because Project Employability combines trarwltional and supported employment, the total cost 
per participant will depend on the length of stay. The cost estimates cover 70 months of 
operation, although most persons had not been enrolled for that long. Thus, costs per 
participant will continue to rise as supported employment services continue to be provided. 

^he figures from Bay State Skills Corporation reflect the experiences of 17 vendors who 
enrolled a total of 306 clients during fiscal vears 1962 and 1963. Average costs per 
participant for these vendors ranged from about $5,500 to $1,700. 
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The STETS evaluation required data on the activities and 
experiences of experimentals and controls from the time of their contact 
with the program and randomization into the research sample to a point 22 
months later. These experiences included labor-market activities, 
participation in training and schooling programs, receipt of transfer 
payments, use of support services, and other activities pertaining to self- 
sufficiency. Data were also required on important demographic and personal 
characteristics. The data for the evaluation had to be collected in a 
standardized manner for all sample members (whether in the experimental or 
control group) over time at appropriate intervals to capture the effects of 
the intervention and key program events, as well as consistently across the 
five demonstration sites. This appendix describes in detail the data 
collection design, and reviews several important methodological and 
fielding issues addressed in this design. It also provides information on 
the results of the data collection effort. 

A. DESIGN OF THE STETS DATA COLLECTION STRATEGY 

The strategy designed by MPR to collect the data for the impact and 
benefit-cost analyses consisted of many components, and was developed from 
our own and others' experiences. In this section, we document the 
background of that strategy, and briefly describe the key components. 

!• Background to the Development of the Data Collection Strategy 

The data-collection approach proposed by MPR to evaluate the STETS 
demonstration was the result of an extensive investigation and evaluation 
of various methods that have been used in previous studies on mentally 
retarded persons and on employment and training programs in general. In 
order to develop a strategy that was likely to yield the best data in the 
most cost-effective manner, MPR conducted the following: 

• A review of literature and available published 
information from other studies^ 



Bibliographies of the important references reviewed in conjunction 
with the design activities can be found in Burghardt, Corson, and Maynard 
(1980) and in the pilot study report (Bloomenthal et al., 1982). 
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• An extensive discussion with expert consultants who had 
collected data from samples of mentally retarded 
persons in previous studies 

• A review of MPR^s and other groups* experiences in 
interviewing populations with similar characteristics 

In other evaluations, similar data collection requirements have 
usually been met by having trained research interviewers administer 
structured interviews to sample members. However, the STETS sample 
members' expected lower-than-average levels of functioning, cognitive 
abilities, and communication skills raised serious concerns about the 
quality of self-reported data. Previous research on this target population 
relied on data from various sources and on a variety of data collection 
techniques. Probably the most common source of data has been "significant 
others" (for example, counselors, job supervisors, and parents and 
guardians) as informants or proxies (see Rusch and Schutz, 1980; Hunt and 
Zimmerman, 1969; Bogen and Aanes, 1975; Lambert and Nicoll, 1976; Ejnnan et 
al. , 1979; and Abramowitz, 1980). In many instances, parents or caretakers 
were expected to articulate the experiences or capabilities of the mentally 
retarded persons who were not interviewed themselves. To date, self- 
reported interviews with mentally retarded persons have been used primarily 
to provide anecdotal details rather than information on major variables for 
statistical analysis (e.g., Wyngaarden, 1981). However, there is evidence 
that individuals who are mildly or moderately retarded are willing and able 
to provide some portion of the data necessary for evaluation research 
through in-person interviews (see, for example, Weinglass, 1980; 
Richardson, 1979; Gollay et al., 1978; Sigelman et al. , 1981a; Birenbaum 
and Re, 1979; and Brolin, 1972). 



efforts, MPR formulated a basic approach, relying primarily on self- 
reported interview data from the mentally retarded sample members. The 
data required for the STETS evaluation were collected through an integrated 
system of data collection efforts, including: 



As a result of our preliminary review of previous data collection 
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• Interviews conducted with the mentally retarded sample 
members and, as necessary, with proxy respondents when 
sample members were unable to provide key data items 



• Corroborating information provided by community service 
agencies with which the sample had contact and which 
were mentioned during the interviews 

• Background information on sample members collected by 
STETS project staff as part of the intake process 

• Program participation data on all experimental group 
members while enrolled in the STETS projects 



The sample member (and proxy) interviews were conducted at four key points 
during the demonstration: 



!• Immediately after random assignment into the sample 
(the baseline interview) 

2. At a point when many experimental group members were 
still actively participating in the STETS project (the 
6-month interview, conducted immediately after 6 
months had elapsed from^ an individual 's random 
assignment into the sample) 

3. At a point when members of the experimental group were 
no longer receiving STETS services (the 15-Tnonth 
interview) 

4. At a point well beyond the end point at which 
individuals stopped receiving demonstration program 
services (the 22-month interview) 

Through our review of previous efforts, we identified several 
critical problem areas in the design of self-reported data collection 
strategies with mentally retarded persons. For example, researchers have 
identified a consistent pattern of acquiescence among mentally retarded 



A two-thirds sample for the 6-month follow-up survey was 

determined to be sufficiently large to detect impacts on sample member 

activities while enrolled in the STETS project, since activities of 

experimental group members at that time would be determined largely by 
participation in STETS. 
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respondents in interview settings (Gerjony ard Winters, 1966; Rosen et al#, 
1977; and Sigelman et al*, 1981b)« This pattern is not surprising given 
general population survey findings which have suggested that acquiescence 
("yea-saying") is more common among both children and less educated adults 
(Lenski et al., 1960; Wells, 1963; and Rothenberg, 19^9). However, this 
problem, which is likely to affect the reliability and validity of self- 
reported data, has been found to be most serious among lower IQ samples and 
is only somewhat less problematic among those with IQ levels that are 
characteristic of the STETS sample (Sigelman et al., 1980) • Interviewer 
behavior, as well as question wording and response formats, have also been 
found to affect the quality of survey data from mentally retarded 
respondents^ As an example, biased responses due to "test anxiety" and a 
heightened desire to please the interviewer are likely scenarios with this 
population (Sigelman et al* , 1983). The design of the STETS interview 
instruments and procedures took into account the sample's expected levels 
of cognitive and communication abilities and interaction skills • 



The following sections review the requirements for each data 



collection instrument developed for the evaluation, as well as the issues 
addressed in their design. 

2# Sample Member and Proxy Interview Instruments 

The baseline and follow-up interviews collected point-in-time data 
on the following: 



• Current employment, job training, and schooling 
activities 

• Current involvement in life-skills training, organized 
recreational activities, counseling, and 
transportation assistance programs 

• Current receipt of transfer payments or benefits, 
including SSI, SSDI, general assistance or welfare. 
Medicaid or Medicare, food stamps , and other 
government or private financial assistance 

• Current living arrangements and residential services 



B*4 



ERIC 




• Current participation in money-handling activities 
including shopping, bill-paying, and banking''^ 

The sample member interview was developed by MPR staff, with 
substantial input from our expert consultants. A number of versions were 
pretested with mentally retarded yo; dults, and although all of the 
pretest interviewees had IQ scores in the range of interest (40-80) we 
concentrated on lower-functioning individuals so as to assess using the 
questionnaire with more problematic respondents. By the time the pilot 
instrument was fielded, over 30 pretest interviews had been conducted with 
a variety of mentally retarded individuals in a number of living and work 
arrangements. The proxy interview was designed to be a close replica of 
the sample member interview. Questions and formats were modified only when 
a question was inappropriate for a nonretarded respondent or to accommodate 
mixed-mode (telephone and/or in-person) administration. Ten pretest proxy 
interviews were conducted. 

Measurement design is especially critical to the success of a self- 
reported interview strategy with a mentally retarded population. For 
example, past experience suggests that questions involving recall are 
likely to present problems for this group. Detailed reports of dates and 
other aspects of past experiences are believed to be especially 
unreliable. Therefore, the STETS interview instruments asked for reports 
of current^ activities only. As described earlier, there is also evidence 
of a consistent pattern of acquiescence among mentally retarded respondents 
in interview settings, although this pattern is less serious among IQ 
populations that are characteristic of this sample. The approach taken in 
the STETS interview was Co follow "yes" answers with questions on the 
details of the activity or experience, to ensure that the initial response 
was not due to acquiescence. 



The baseline interview also contained information on the work 
history and occupation of those with whom the sample member had lived while 
growing up. 
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Quantitative concepts such as those involving time and money are 



especially difficult for mentally retarded persons. Mentally retarded 
individuals' knowledge about one particular area of interest to the STETS 
evaluation — their financial situation (sources and amounts of income, 
expenses, and assets) — also varies greatly according to both their level of 
independence and their cognitive abilities. There may be fairly large gaps 
in the knowledge of some sample members in terms of the details of their 
financial status, depending on whether they handle their own finances. In 
the STETS interview instrument, questions on earnings and the receipt of 
transfer payments, as well as on other quantitative concepts, were broken 
down into simpler subquestions . For example, earnings on a job was 
determined by asking for the rate of pay, the frequency of receiving pay, 
and the usual (or last) amount received. If the rate of pay was unknown, 
the other questions were used to construct it. 



can affect the quality of survey data. One major concern was the number 
and direct iveness of probes. The mentally retarded are likely to be unsure 
of their answers and might initially respond "don't know" to many 
questions, both factual and attitudinal. However, excessive probing may 
provoke biased responses due to "test anxiety" and a heightened desire to 
please the interviewer. Therefore, the STETS interview instrument 
specified the exact number and type of probes to be used by the interviewer 
on items thought to be particularly likely to require probing. 

3. Agency Service Interview and Coding Form 

During the interviews with primary and proxy respondents, the 
following entities were identified and assigned a unique identifying 
code: community service providers, employers, and resia^»ntial service 
agencies. Interviewers also obtained from the respondents sufficient 
information to contact the organization, and then attempted brief telephone 
interviews with knowledgeable informants at each organization. These 



These questions were among those for which missing or inconsistent 
responses indicated the need for a proxy respondent. 



Both interviewer behavior and question wording and response formats 
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interview instruments were designed to determine the types of clients 
served and the mix of services provided if the organization was in fact a 
service agency. Private employers and other organizations which did not 
provide services were asked an abbreviated set of questions. Sample 
members who had named the organizations were not identified, and no attempt 
was made to collect individual^level service data. Agency-levei data were 
used to corroborate sample member reports on services received, including 
the type of residential arrangements, schooling, and employment training. 
These data were also used to estimate the cost of services received by the 
sample, either as an alternative to or in addition to STETS, for 
application in the benefit-cost analysis. 

4. Application/Enrollment Form 

As part of the intake process, STETS project staff completed 
application/enrollment forms for all sample members. The application/ 
enrollment form summarized information collected from a number of sources — 
Che applicant, parents or guardians, referral agency staff or records, and 
records or reports from other agencies. It contained information 
certifying the applicant's eligibility for the STETS demonstration (date of 
birth, IQ score and documer.tation, recent work history, and secondary 
handicaps), as well as other background information, including living 
arrangements, parental background, and history of schooling, training, and 
employment. The information from the application/enrollment form provided 
data pertaining to two important topics: (1) the baseline experiences and 
characteristics of the sample prior to the receipt of any STETS services, 
and (2) the pest education and employment services received by sample 
members. Both topics were critical to the analysis. Data pertaining to 
both could not be gathered reliably in the first (baseline) survey, because 
the time required for assignment and contact attempts meant that baseline 
interview d£ ta from the sample members were obtained at an average of 
approximately thirty days after random assignment, when most experimental 
group members would have begun to participate in the STETS program. 
Moreover, such data could not be sought retrospectively in the first 
interview, because of the limited ability of the sample to report details 
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of their past experiences accurately (the interview asked about current 
activities only). 

The staff of MDRC and MPR met several times in the process of 
developing the application/enrollment form and drew upon an early review of 
draft forms by STETS operators and some of their referral agencies. The 
form underwent several modifications while used during the early months of 
program intake. The initial instructions to the sites were to complete the 
entire form regardless of the data source; later, the emphasis shifted to 
completing most of the data items from records. 

5. STETS Participation Data 

The application/enrollment form for each experimental group member 
initiated an entry for that individual in the Management Information System 
(MIS) maintained by MDRC for the STETS projects. During the period of the 
participation of experimental group members in the STETS demonstration, 
information on each individual was provided on a monthly basis to the MIS 
database. IMs information included the individual's current status in the 
project, placement da to .n training and permanent jobs, reasons for changes 
in prograTi status (from the Monthly Status Change Form), the number of days 
actively involved in STETS, and the hours scheduled and actually attended 
in various types of demonstration activities (from the Monthly Activity 
Form). These data were used in the Impact and benefit-cost analyses to 
determine the length of program participation for individuals and the level 
of STETS services provided to them. 

B. PILOT STUDY 

Following the design period, the data collection for the STETS 
evaluation proceeded in two phases. Between November 1981 and January 
1982, a pilot study, including interviews with sample members and proxy 
respondents, was conducted with treatment and control group members in 
three sites. The pilot study results were used to modify the instruments 
and procedures. The second phase of the data collection began in April 
1982, when fielding began in all five sites, and continued until October 
1984, when the last follow-up interviews were conducted. This section 

B.8 



ERIC 



276 



briefly reviews the results from the pilot study; further details on the 
procedures and findings of this study are available in a separate report 
(Bloomenthal et al., 1982). 

1. Study Overview 

A pilot study was undertaken to inform final design decisions about 
the best source(s) of data for the evaluation, given the uncertainty about 
the quality of self-reported interview data from mentally retarded sample 
members. The pilot study entailed conducting data-collection activities 
with the research sample which was enrolled between November 1, 1981 and 
January 31, 1982 in Cincinnati, New York, and Tucson. The pilot-phase 
design called for interview attempts with all research sample members and 
an identified proxy for each respondent. Application/enrollment forms for 
each sample member were received, and the data were entered for data 
comparisons. Data from a total of 104 sample members were included in the 
pilot study analysis. The study also investigated the availability and 
quality of data from offirial records. 

The pilot study confirmed the ability of most of the STETS sample 
to respond to research interviews and generally to provide complete and 
accurate data on themselves. Records and proxy respondents were not found 
to be superior sources, in terms of either completeness or data quality. 
The key findings of the study are summarized in the following sections. 

2. Sample Completeness 

High response rates with both sample members (95 percent) and proxy 
respondents (99 percent) indicated that the interview strategy could 
provide baseline and follow-up data on virtually all sample members. The 
application/enrollment form also provided a high degree of sample 
completeness, but was available only for certain baseline data items. 

3. Data Completeness 

Little or no missing data occurred for many of the variables in the 
pilot study data, including the education and training variables—both 
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current (from the interview) and prebaseline (from the application/ 
enrollment form) living arrangement and family composition (from all three 
sources), and other IJ ving-skills activities (e.g., independence in ncney- 
handling from the interviews). Other types of variables had greater levels 
of missing data, regardless of the source. Transfers-program use was the 
most striking case. Some aspects of labor-market performance, particularly 
earnings, also suffered from substantial missing data from both the sample 
member and the proxy interviews. 

The missing interview data found during the pilot study followed 
the patterns that were expected from a review of the available literature, 
the experience of consultants, and pretest experiences. Key areas were 
those that involved money, particularly the amounts of earnings and the 
receipt and amount of transfer payments. In the area of transfers (both 
cash and in-kind), patterns of nonresponse by sample members led us to 
believe that a "don't know" response might indicate a reluctance to say 
"no" when the question seemed ambiguous. From these patterns, we were able 
to design an appropriate rule for using proxy interviews which 
significantly decreased the amount of missing data. 

The missing data encountered on the application/enrollment form 
were due to a variety of problems, and there were significantly more 
missing data on the form than in the interviews. However, the form did 
provide adequate completeness on some key data items (e.g., IQ) that were 
not available from other sources. 

4. Data Consistency 

An analysis of data consistency across sources, together with the 
analysis of completeness, enabled us to draw inferences about quality. 
Generally, the consistency between sample member and proxy pilot study data 
was quite high. Where reporting differences did appear, there were 
indications that any errors underlying the inconsistencies were as likely 
to come from proxy respondents as from sample members. 
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C. FIELD PROCEDURES 

This section describes the procedures used to implement the data 
collection design. These field procedures were initially used during the 
pilot phase, and were then modified and extended to the second phase of 
data collection. The procedures discussed here include interviewer 
recruitment and training, interviewing, supervision, and quality control. 

!• Interviewer Recruitment and Training 

The interviewing staff was critical to the success of the STETS 
evaluation. Field interviewers were responsible for implementing the data 
collection design through interactions with the mentally retarded sample 
members and their parents, guardians, and other caregivers, with the STETS 
project staff, and with directors of the many community agencies from which 
the sample received services. They had to maintain detailed confidential 
records to help locate sample members for the follow-up Interviews. 
Because of the small sample size, only one interviewer was hired in most 
sites, and that person had to be able to carry out all field data 
collection tasks independently, without face-to-face daily supervision. 
The importance of field staff to the ev-iluation dictated very careful 
interviewer recruitment and training efforts. 

Recruitment. Applicants for interviewer positions were recruited 
primarily through classified advertisements in the laajor newspaper in each 
of the demonstration sites or through recommendations from the STETS 
operators in each site. Applicants who responded to our newspaper 
advertisements and who had relevant experience, both in working with the 
mentally retarded or similar populations and in performing research 
interviewing, were contacted by telephone. An outlim; that was followed 
during this telephone conversation gathered more details on the quality and 
extent of the interviewer applicants' experience and assessed their 
willingness to undertake the work. The following types of experiences and 
attitudes were assessed: 

• A willingness to be a data collector without being 
able to offer advice, referrals, or services 
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• An interest in the study content and an understanding 
of the general research objectives 



• Experience in setting up and maintaining files and 
records and with regular reporting/monitoring 
procedures 

• Experience in arranging a schedule of appointments by 
telephone and in locating dif f icult-to-locate persons 

• Having a home office already established, with work 
and storage space as well as a telephone 



Applicants were also asked about their current employment or other 
commitments and whether they had regular use of an automobile. These 
telephone screening interviews were reviewed, and the best set of 
applicants were contacted for in-person interviews. 

Senior survey staff traveled to the sites and conducted in-depth 
interviews with the selected applicants. In-person recruitment interviews 
were scheduled for two days — the first day involving formal interviews with 
applicants, and the second day involving visits to the homes of the top two 
to four candidates. The formal interviews were designed to provide a more 
detailed follow-up on the relevant experience and background of the 
applicants by questioning them about areas of concern that had been 
identified In the telephone screening. The applicant's general style was 
also crucial, since his or her role would involve contact with the STETS 
program and other local agencies, with parents and guardians, and, of 
course, with the mentally retarded young adults themselves. Applicants 
were £ilso asked to conduct a brief mock interview with an MPR staff member 
who acted as a respondent. 

The home visit allowed the recruiter to talk with and observe the 
applicant In a more relaxed setting, as well as to answer any additional 
questions he or she might have about the job, to obtain a»re de-tiails on 
potential issues of concern (flexibility, travel time, other commitments, 
etc.), and to look over the available office space. Extensive reference 
checks with recent past employers were also part of the final a'ecision 
process. 
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The STETS Interviewers * Four pilot study interviewers were hired- 
one each in Cincinnati and Tucson and two in New York* One New York 
interviewer was a woman with an educational background in clinical 
psychology, who had experience in one-on-one tutoring and had worked in a 
group home with mentally retarded young people* The other New York 
interviewer was a man who had interviewing experience on an MPR study of 
the impaired elderly. The woman hired in Tucson was a tutor and counselor 
with the mentally retarded in the same agency which was conducting the 
STETS program. The Cincinnati interviewer was a woman with a background in 
volunteer work and paid employment with the mentally retarded, and who had 
helped compile a local directory of services for this group. 

After the pilot study, the female New York interviewer was retained 
for the full study, and the Cincinnati interviewer was replaced by a woman 
who had worked for the local STETS host agency prior to taking maternity 
leave. She had had extensive experience in counseling and training 
mentally retarded persons. Interviewers were also hired at that time in 
the Los Angeles and St. Paul sites. Two women were recruited in Los 
Angeles to cover the large catchment area of that STETS project. One of 
the Los Angeles interviewers was a woman wJ.th teaching experience and who 
was a MPR interviewer. The other woman was an interviewer with personal 
experience in working with mentally retarded young adults. The St. Paul 
interviewer was a woman with experience in both counseling and research 
interviewing. 

There was virtually no interviewer attrition after the pilot 
study. One Los Angeles interviewer was laid off in the summer of 1982 due 
to the lack of work, and the remaining interviewer continued with the 
project to its completion. The New York interviewer left the project in 
February 1983 to take a full-time job as a counselor in a community 
residential program for mentally ill clients. She was replaced by an 
experienced interviewer, a man who had worked with MPR on a number of youth 
employment studies. Thl« interviewer underwent thorough individualized 
training, assisted by the outgoing New York interviewer. He and the other 
interviewers continued on the project until its completion in the fall of 
1984. 
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Training . The role of the interviewer involved a complex set of 
tasks, as well as interaction with different individuals and agencies in 
the process of completing a single interview assignment. The training 
sessions were necessarily lengthy and intensive, involving practice 
sessions and feedback by MPR staff, with special tutoring as necessary. 

Training was conducted during four-and-a-half-day sessions 
(December 7-11, 1981, for the pilot study, and April 12-16, 1982, for the 
ongoing study) held at the MPR offices in Princeton. Two manuals were 
prepared for these sessions, covering the full range of field issues and 
activities — Interviewers ' Procedures Manual aid Instrument Training 
Manual; Primary and Proxy Instruments and Agency y.og . 

These manuals served as the basis for the training sessions, 
providing detailed information on all aspects of field procedures and 
questionnaire usage. A considerable portion of the training was devoted to 
the practical use of the various forms and instruments. MPR survey staff 
held several round-table and one-on-one mock interviews with the trainees, 
and observed and commented on all aspects of questionnaire administration 
and field procedure . Interviewers whc had pilot study field experience 
assisted in the training, sharing their experiences and demonstrating 
effective techniques for contact attempts, interviewing, and record 
keeping. 

One of the most useful activities during training was the 
interviews conducted by the interviewer trainees with local mentally 
retarded young adults, most of whom were at the lower end of the STETS 
eligible range in tenns of IQ and functional ability. These "real-world" 
practice interviews gave the interviewer trainees confidence in their 
ability to handle field situations with mentally retarded respondents 
before they were in the field. Moreover, MPR survey and research staff who 
observed the Interviews had the opportunity to provide better assessments 
of the strengths and weaknesses of the individual interviewers. The round- 
cable debriefing held afterward was a time to share problems, discuss 
possible solutions, and provide feedback on interviewer performance. 
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Mid-Project Interviewer Conference . On January 24-25, 1983, an 
Interviewer conference was held In Princeton to review changes In project 
schedules and procedures. This conference was attended by all the survey 
staff, with the exception of the Tucson interviewer, who participated, as 
possible, by telephone. Four main topics were discussed at the conference: 

1. Administering rhe interviews and further detailed 
instructions, based on interviewer experiences and 
questions 

2. Contacting respondents for the follow-up interviews, 
particularly the one-third of the sample who were not 
assigned the 6-month follow-up 

3. Conducting the agency services interviews and coding 
the services log 

4. Administrative procedures and Issues pertaining to 
interviewing assignments and pay schedules 

The conference provided an excellent forum for research, survey, and field 
staff to review the goals of the STETS evaluation, discuss data quality 
Issues, and resolve field problems. A member of the MDRC staff also 
attended the conference. 

2. Interviewing 

Interviewing activities in the field for the baseline pilot study 
began on December 14, 1981, and ended on January 31, 1982. The second 

phase of interviewing began on April 19, 1982, after the final data 

collection design had been approved by MDRC, additional interviewers were 

recruited, and interviewer training had been completed. The field period 
ended on October 31, 1984. 

Assignments and Contact Attempts . During the fielding of the 
baseline Ir.terviews, weekly assignments (when Intake warranted them) vere 
sent to Interviewers from the logs kept of the applicants who had been 
randomly assigned at each site. Interviewers were sent the name and 
identification number of each new sample member. Interviewers were 
expected to pick up from the STETS program the consent materials and 
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application/enrollment forms that were necessary to begin scheduling 
interviews* 

At the time of application to the STETS projects, project staff 
explained the conditions of participation in the demonstration and the 
associated evaluation — in particular, random assignment and periodic 
research interviews. Before applicants could be enrolled in the research 
sample and, if experimental group members, provided with demonstration 
services, their written consent had to be obtained and co-signed by a 
parenc or guardian if necessary. The baseline interview could not be 
conducted until the research interviewer had obtained this consent form 
from the STETS project. No further written consent for the follow-up 
interviews was obtained from the primary sample members. However, at the 
time each interview was conducted, the interviewers answered any questions 
and explained the voluntary nature of the interview and the confidentiality 
of information obtained during the interview. 

Information from the application/enrollment form was used to 
prepare advance letters and other material for interviewing contacts. 
These letters were followed by telephone calls to the primary respondents 
to arrange an appointment for an in-person interview. At that time, the 
interviewer also spoke with the parent, guardian, houseparent, or otner 
responsible person if the primary respondent was not living Independent- 
ly. If there was no telephone number on the application/enrollment form or 
if the contact information was no longer valid, the interviewer made 
personal visits to the home and/or initiated search procedures until the 
primary respondent could be located or a final noncompletion status 
assigned. Interviewers kept detailed records on ^11 contact attempts for 
every sample member during both baseline and follow-up fielding. 

The sample member was interviewed at the scheduled time in the home 
if possible or someplace else where the sample member would feel at ease. 
Upon completion of the interview, the sample member was given a $5 cash 
respondent payment. These payments were well received and seemed to 
contribute to the respondents' willingness to be interviewed. Interviewers 
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obtained receipts for the respondent payments and submitted these receipts 
along with their other interviewing expenses. Proxy respondents did not 
receive any payments. 

During the follow-up interviewing, assignment sheets which listed 
the name and identification number of the sample members who were eligible 
for an interview during the upcomii^ month were sent to the interviewers at 
the end of the preceding month. Advance letters, telephone calls, and in- 
person attempts were used to contact and interview the follow-up sample. 
During the follow-up interviewing, interviewers were allowed to use their 
own judgment in deciding whether to send advance letters to parents when 
the sample member did not live at home. 

Identifying and Int erviewing Proxy Respondents . Critical items on 
the sample member interview were used to determine the necessity of 
conducting a proxy interview. These included items which identified the 
sample member's major activity (employment, training, or schooling), hours 
and earnings of any employment-related activities, and the receipt or 
amount of cash benefits from government transfer programs or other 
sources. Based on specific instructions in the sample member interview, 
interviewers noted cases in which the sample member was unable to provide 
the required information, where the information was inconsistent 
(specifically, when reported SSI benefit amounts exceeded the maximum 
possible in the state of residence), and when the sample member appeared 
cjenerally confused or was unintelligible. In these cases, an appropriate 



Sample members were eligible for the follow-up interviews only if 
their completed baseline interview had been received in Princeton. The 6- 
month follow-up interview was attempted only for a random two-thirds of the 
futl sample, indicated at the time of randomization. 
2 

A set of questions was included as the first module in the 
interview to determine the sample member's name, address, telephone number 
and age. Besides providing a non-threatening introduction to the inter- 
view, the original intent of this module had been to identify respondents 
who could not provide this basic information, as a way to screen out those 
who were unable to complete the interview. However, most sample members 
could answer all these items correctly, thus making it useless as an early 
screen for those who would need proxy respondents. IQ score was also found 
to be an inadequate predictor of the necessity for a proxy interview. 

B.17 



ERIC 



285 



proxy respondent was selected from those who were named during the sample 
member interview as providing significant services or support. The proxy 
respondent was selected in the following order of priority: 



!• A live-in parent or relative who gave help with 
financial management 

1. Any other person who gave help with financial 
management 

3* A live-in parent or relative, when no help with 
financial management was received 

4# A social worker or caseworker 

5# Someone whom the sample member indicated was 
knowledgeable, when no other criteria were met 



The proxy respondent was interviewed immediately following the 
sample member interview if possible; if not, further contacts were made 
until an interview could be scheduled and completed. Additional letters 
were sent to proxies who had not been contacted during the initial contact 
process. After the pilot study (in which in-person interviews were 
required), interviewers could conduct the proxy interview over the 
telephone, if necessary. 

Field Editing and Document Transmittal. After the interviews were 
completed, interviewers edited all the instruments and forms. Marginal 
notes were to be added as necessary to explain special circumstances or to 
provide details on ambiguous situations. Agency names mentioned during the 
sample member or proxy interview were entered onto an agency log, and code 
numbers were assigned and transferred to the interview documents. Agencies 
were contacted and asked to describe their services in order to complete 
the agency log form. 

Interviewers maintained files for each sample member in their 
site. These files included contact worksheets which contained information 
that would be useful in later contact attempts. Such information included 
the names, addresses, and telephone numbers of friends or relatives who 
were likely to know where the sample member could be located, and the 
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agencies or organizations in which he or she had been active. These notes 
also included information on any problems encountered during the interview 
administration, such as very protective parents, soeech impediments, or 
emotional upsets* 

Documents were sent to MPR in Princeton in two separate packets — 
one for the confidential material (application/enrollment form, signed 
consent form to participate in the research, contact sheet, and signed 
release to interview the proxy), and another for the completed interview 
instruments themselves. Interviewers and MPR staff kept independent 
records of assignments, completions, and mailings, which were reviewed and 
reconciled weekly. Interviewers also reported their time and expenses on a 
weekly basis » 



3. Supervision and Quality Control 

Once received in Princeton, interview documents (interview 
instruments, contact sheets, release forms, and agency logs) were logged in 
and edited by an experienced quality control clerk. Interviewers were 
trained to make extensive marginal notes on any circumstances that would 
affect how responses were coded during the interview. The quality control 
clerk carefully reviewed all such marginal notes when evaluating the 
appropriateness of the coded responses. Items on the documents for which 
responses were missing, ambiguous, or contradictory to other responses were 
flagged, and these issues were discussed with the interviewer during the 
next telephone call. Issues that could not immediately be resolved by the 
interviewer were assigned for a call-back by the inter\'iewer to the 
respondent. As necessary, memoranda v/ere circulated to the quality control 
and interviewing staff who were responsible for reviewing recent policy 
decisions that affected their work. These memoranda often discussed how to 
handle unusual situations encountered during interviewing or brought to 
light during quality control editing. 

The quality control clerk conducted verification interviews with a 
random subsample of completed sample member and proxy interviews. The rate 
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of verification interviews assigned was greatest for new interviewers 
during the first several weeks of work, and tapered off to a less frequent 
but still regular schedule thereafter* Items on the verification interview 
confirmed that the interview had be*^ii conducted, asked some basic factual 
information about the sample member (which was compared with interview 
data), ascertained whether there had been any problems with the 
interviewer's conduct, and verified that the $5 respondent payment: had been 
made. A total of 254 verification interviews were completed throughout the 
entire study period. During the pilot field period, verification 
interviews uncovered problems with two interviewers' work, which were then 
rectified at that time: the interviewers were terminated from the 
project. No problems of any kind were encountered either with the work of 
the remai^ang interviewers or with the work of the interviewers hired later 
in the project. 

Interviewers had a regular weekly reporting schedule with tht ''^^R 
survey manager. During these telephone reports, assignment logs were 
updated with new final statuses. Reports of interim statuses and mailings 
were also made on a case-by-case basis. Additional telephone calls, 
initiated either by the interviewer or by MPR survey staff, were made to 
clarify contact or interviewing situations, to review changes in documents 
or procedures, and to resolve any discrapancies or errors in the 
interviews. These calls were made very frequently at the beginning of the 
field period, as interviewers confronted new situations and as MPR project 
staff made necessary modifications to procedures based on unanticipated 
circumstances. As fielding proceeded, the calls were made less frequently. 

The weekly telephone reports from the field formed the basis for 
the field-status reports monitored by MPR project staff and provided 
regularly to MDRC. The receipt of materials in the Princeton office and 
the progress of these materials through quality control and data entry were 
also rr >rded. 

Periodic site visits with each interviewer by MPR survey and 
research staff were conducted throughout the field period. During these 
visitB, interviewer records and files were reviewed , interviewing or 
record-keeping; problems or concerns were discussed, and an interview with a 
sample member was observed. 
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The STETS program operators were responsible for completing the MIS 
application/enrollment forms. Completed application/enrollment forms were 
sent to MDRC for quality control and site call-backs, where indicated. 
After problems were resolved, the forms were sent to MPR for /additional 
quality control, coding tor selected research questions, and data entry. 

C. INTERVIEWING RESULTS 

Overall, the data collecMon strategy using interviews with sample 
member and proxies was very successful, achieving both high completion 
rates and data of good quality. This section documents the interviewing 
results for each interview wave by type of respondent (sample member and 
proxy), site, and research status (experimental and control). In addition 
to the final status of each interview attempt, we present information on 
the completed interviews— elapsed time between assignment and completion, 
length of interview, location of interview, anr other details of the 
interview process. The completeness of the resulting data set is also 
discussed. 



!• Interviews with Sample Members 

Tables B.l through B.4 present the final statuses and response 
rates for interview assignments with the mentally retarded sample members 
at each wave. Response rate is defined as the number of completed 
interviews divided by the total sample assigned less those ineligible to be 
interviewed because they were incarcerated or deceased, or had moved out of 
the study area. These interviewing results show consistently high 
completion rates of interviews across sites and between the research 
statuses. At baseline, 455 interviews were completed, for an overall 
response rate of 97.6 percent (see Table B.l). There was only a small 
difference (3 percentage points) in the overall response rate between the 
experimental group and the control group (99. 1 versus 96. 1 percent) , 
although these differences varied by site. In all cases, the response rate 
was greater for the experimental group. There were no substantial 
differences in the overall response rates among the sites, the largest 
being only 5 percentage points between Los Angeles and St. Paul. 
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TABLE B.1 



SAMPLE MEMBER BASELINE INTERVIEW 
FimL STATUS AND RESPONSE RATE BY SITE AND RESEARCH STATUS 

(Percent) 



Final Status 



Total 
Assigned 



Complete/ Sample 
Partial Member 
Complete Refused 



Parent/ Non- 

Guardian English- Incar- 

Refused Speaking cerated 



Moved 
Out of 

Area Deceased 



Unable to 
Locate or 
Contact 



Cincinnati ^ « a 

Experimental 48 97.9 0 ^ £ ? £ 

Control 47 95.7 2.1 0 0 0 0 

Total 95 96.8 1.1 0 0 0 0 



Los Angeles 

Experimencal 5?. 100.0 0 0 0 

Control 50 88.0 4.0 0 4.0 

Total 102 94.1 2.0 0 2.0 



0 
2.0 
1.0 



0 

0 
0 



0 
0 
0 



0 
0 
0 



2.1 
2.1 
2.1 



0 
2.0 
1.0 



97.9 
95.7 
96.8 



100.0 
89.8 
95.0 



York ^ ^ ^ 

Experimental 58 98.3 0 ^ £ 2 £ 

Control 57 98.3 1.7 0 0 0 0 

Total 115 98.3 0.9 0 0 0 0 



0 
0 
0 



1.7 

0 

0.9 



98.3 
98.3 
98.3 



St. Paul 

Experimental 27 100.0 0 

Control 27 100.0 0 

Total 54 100.0 0 



0 
0 
0 



0 
0 
0 



0 
0 
0 



0 
0 
0 



0 
0 
0 



0 
0 
0 



100.0 
100.0 
100.0 



Tucsoir* 

Experimental 51 100.0 0 0 0 

Control 50 98.0 0 2.0 0 

Total 101 99.0 0 1.0 0 



0 
0 
0 



0 
0 
0 



0 
0 
0 



0 
0 
0 



100.0 
98.0 
99.0 



Total ^ ^ 

Experimental 236 99.1 0 0 0 0 0 

Control 231 ?5.7 1.7 0.4 0.9 0.4 0 

Total 46 7 97.4 0.9 0.2 0.4 0.2 0 



0 
0 
0 



0.8 
0.9 
0.9 



99.1 
96.1 
97.6 



defined as the nunfcer of completed interviews divided by the total nurrter assigned less those sample members who were 
incarcerated or deceased, or had moved out of the study area. 
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SAMPLE MEMBER 6.M0NTH INTCRVIEW 
FIHM STATUS AND RESPONSE RATE BY SITE AND RESEARCH STATUS 

(Percent) 



Total 
Assigned 



Complete/ Sample Parent/ 
Partial Member Guardian Incar- 
Complete Refused Refused aerated 



Final Status 
Moved 
Oat of 

Area Deceased 



Unable to 
Locate or 
Contact 



Rcsi 



pogsc 



CindfiMti 




















Experimental 


32 


90.6 


0 


0 


6.3 

0 

3.2 


3.1 

0 

1.6 






100.0 
96.7 
98.3 


Control 
Total 


30 


96.7 


0 

U 


0 
0 


0 
0 
0 


0 
3.3 
1.6 


Los Angeles 




















Experimental 

Control 

Total 


30 

^ 


88.2 
90.0 

fl9 1 


5.9 
3.3 

•f. / 


0 
0 
0 


2.9 
3.3 
3.1 


2.9 

0 

1.6 


0 
0 
0 


0 
3.3 
1.6 


93.7 
93.1 

93 A 


Nm York 




















Experimental 


37 


97.3 


2.7 


0 


0 


0 






97.3 
97.ti 
97.3 


Control 
Total 


38 
75 


97A 
97.3 


0 

1.3 


2.6 
1.3 


0 
0 


0 
0 


0 
0 
0 


0 

0 
0 


St. Paul 




















Experimental 


18 


100.0 


0 


0 
0 
0 


0 


0 




0 
0 
0 


100.0 
100.0 
100.0 


Control 
Total 


18 
36 


100.0 
100.0 


0 
0 


0 
0 


0 
0 


0 
0 
0 


Tucson 




















Experimental 


3<^ 


97.1 


0 


0 


0 


0 


2.9 

0 

1.5 




100.0 
96.9 
98.5 


Control 
Total 


32 
66 


96.9 
97.0 


0 
0 


0 
0 


0 
0 


0 
0 


0 
3.1 
1.5 


Total 




















Experimental 

Control 

Total 


155 
148 
303 


95.9 
95.0 


1.9 
0.7 
1.3 


0 
0.7 
0.3 


2.1 
0.7 
1.3 


1.3 

0 

0.7 


0.6 

0 

0.3 


0 
2.0 
1.0 


98.0 
96.6 
97.3 
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TABLE B.3 



SAMPLE MEMBER 13-MONTH IMTERVIEW 
FimL STATUS mo RESPONSE RATE BY SITE mO RESEARCH STATUS 

(Percent) 



CindMiati 

Experinental 

Control 

Total 



Angeles 

Experlnental 

Control 

Total 

m Yoifc 

Experinental 

Control 

Total 



St. P^l 

Experlnental 

Control 
Total 

Tucson 

Experimental 

Control 

Total 

Total 

Experimental 

Control 

Total 



Total 
Assigned 



Complete/ Sample Parent/ 
Partial Member Guardian Incar- 
Comple te Refused Refused ce rated 



Final Status 



^7 

92 



52 
96 



57 
56 
113 



27 
27 
54 



51 
49 
100 



234 
221 
455 



87.2 
97.8 
92.4 



90.4 
88.6 
89.6 



87.7 
85.7 
86.7 



96.3 
100.0 
98.1 



96.1 
89.8 
93.0 



91.0 
91.4 
91.2 



2.1 
2.2 
2.2 



1.9 

0 

1.0 



5.3 
1.8 
3.5 



0 
0 
0 



0 
0 
0 



2.1 
0.9 
1.5 



0 
0 
0 



0 
0 
0 



3.5 
7.1 
5.3 



0 
0 
0 



0 
0 
0 



0.9 
1.8 
1.3 



0 
0 
0 



0 
4.5 
2.1 



0 
0 
0 



0 
0 
0 



2.0 
1.0 



0 

1.4 
0.7 



Moved 
Out of 

Area Deceased 



6.4 

0 

3.3 



7.7 
2.3 
5.2 



0 
3.6 
1.8 



0 
0 
0 



2.0 
6.1 
4.0 



3.4 
2.7 
3.1 



0 
0 
0 



0 
0 
0 



0 
0 
0 



3.V 
0 

1.9 



2.0 

0 

1.0 



0.9 

0 

0.4 



Unable to 
Locate or 
Contact 



4.3 

0 

2.2 



0 
4.5 
2.1 



3.5 
1.8 
2.7 



0 
0 
0 



0 
2-0 
1.0 



1.7 
1.8 
1.8 



RespoQse 
Rate^ 



93.2 
97.8 
95.5 



97.9 
95.3 
96.6 



87.7 
88.9 
88.3 



100.0 
100.0 
100.0 



100.0 
98.0 
99.0 



95.1 
95.3 
95.2 



^f ined as the mnber of completed interviews divided by the total nufiber assigned less those sample menbers who were 
incarcerated or deceased, or had moved out of the study area. 
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TABLE B.4 
















SAMPLE MEMBER 22-MOKTH INTERVIEW 
Fim. STATUS AND RESPONSE RATE BY SITE AND RESEARCH STATUS 

(Percent) 


















Final Status 










Total 
Assigned 


Complete/ 

Partial 

Lompiece 


Sample 
Member 
Kef used 


Karent/ 

Guardian 

Refused 


Incar- 
cerated 


Moved 
Out of 

Area Deceased 


Unable to 
Locate or 
Contact 


RespoQSti 
Rate 


CindiiMti 

Experimental 

Control 

Total 


^5 
92 


85.1 
86.7 


2.1 
4.4 


0 
0 


2.1 

0 

1.1 


4.3 
6.7 
5.4 


0 
0 
0 


6.4 
2.2 
4.3 


90.9 
92.9 
91.9 


Los Angeles 

Experimental 

Control 

Total 


^ 
96 


92.3 
86.4 

flO £. 
07.0 


1.9 

0 

1 A 

1.0 


0 
0 
0 


0 

2.3 
1.0 


5.8 
6.8 
6.3 


0 
0 
0 


0 
4.5 
2.1 


98.0 
95.0 
96.7 


Nbv Yoric 

Experimental 

Control 

Total 


57 
56 
113 


84.2 
82.1 

OJmC. 


5.3 
1.8 


3.5 
5.* 
4.4 


0 

1.8 
0.9 


0 

3.6 
1.8 


0 
0 
0 


7.0 
5.4 
6.2 


84.2 
86.8 
85.5 


St. Pml 

Experimental 

Control 

Total 


27 
27 
5* 


96.3 
100.0 
98.1 


0 
0 
0 


0 
0 
0 


0 
0 
0 


0 
0 
0 


3.7 

0 

1.9 


0 
0 
0 


100.0 
100.0 
100.0 


Tucson 

Experimental 

Control 

Total 


51 
^9 
100 


92.2 
89.8 
91.0 


0 
0 
0 


0 
0 
0 


0 

2.0 
1.0 


2.0 
8.2 
5.0 


2.0 

0 

1.0 


3.9 

0 

2.0 


95.9 
100.0 
97.8 


Total 

Experimental 

Control 

Total 


234 
221 
455 


89.3 
87.8 
88.6 


2.1 
1.4 
1.8 


0.9 
1.4 
1.1 


0.4 
1.4 
0.9 


2.6 
5.4 
4.0 


0.9 

0 

0.4 


3.9 
2.7 
3.3 


92.9 
94.2 
93.5 


^flned as the 
Incarcerated or 


nuifcer of completed Interviews divided by the total nurtfcer assigned less those sample 
deceased, or had moved out of the study area. 


members who were 
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The response rates for the follow-up interview waves ^ were also 
high, as shown in Tables B.2 through B.4. For example, the overall 22- 
month response rate was 93.5 percent, with virtually no difference by 
research status (92.9 percent for the experimental group and 94.2 percent 
for the control group). Note that in the follow-up interviews the control 
group was somewhat more willing to be interviewed than was the experimental 
group, possibly because members of the control group who were not at all 
interested in the research or who were difficult to interview had not 
completed a baseline. Mor.:over. ancedotal reports from the interviewers 
suggest that a small number of experimental group members beca.-<e 
disillusioned with the program and refused to take part in further research 
interviews. Site differences in response rates for the follow-up 
interviews were also generally small, although the New York site had higher 
rates of refusal and unable-to-lccate final statuses in the later waves 
(15- and 22-month follow-ups) than the other sites. By the 22-month, the 
New York response rate was 85.5 percent overall. 8 percentage points below 
the overall rate, but still high in comparison with most longitudinal 
studies. 

The excellent overall response rate by sample members (and. as 
shown later, by proxy respondents) is probably due to a combination of 
factors: the explanations of the research given by the STETS intake 
counselors, the advance letters sent by the interviewers to both sample 
members and their parents or guardians, and the efforts that interviewers 
mad? to explain the study during their contact with respondents. The 
respect and consideration that the interviewers showed toward sample 



Follow-up interviews were assigned only for those sample members 
who had completed the baseline interview. In addition, the 6-raonth follow- 
up included only those sample members who were in the two-thirds subsample 
determined at randomization. 

2 

By comparison, the completion rate for the youth sample in the 
national Supported Work demonstration 18-inonth follow-up was 74 percent, 
with a 6 percentage point difference between experimental and control 
groups. In the New York Supported Work site, the completion rate was 81 
percent for the experimental group, and 67 percent for the control group 
(see Jackson et al.. 1979). 
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members in giving them an opportunity to speak for themselves may have also 
encouraged response. The $5 respondent payments were well received and 
also seemed to contribute to respondents' willingness to be interviewed. 

Table B.5 presents the elapsed time in days between assignment and 
the date of completion, by interview wave and research status. The 
baseline interview with the sample member was completed just over an 
average of one month after random assignment. Baseline interviews were 
assigned weekly or biweekly to interviewers from the central office records 
of randomization. Once the assignment sheet reached the interviewer, he or 
she had to visit the local STETS office to pick up a copy of the 
application/enrollment form, which contained contact information and the 
signed consent form. The advance letters were then mailed to the primary 
respondents and their parents, and, after the letters were received, 
telephone and in-person contact attempts were made to schedule 
appointments. The necessary delays in executing these steps, and the fact 
that there was a small backlog of sample members, both in the baseline 
pilot phase and once ♦'^'^ full baseline interviewing began, meant that 
almost 32 days elapsed ocCween assignment and completion at baseline. On 
average, members of the experimental group were interviewed about 3 days 
later than control group members who were assigned on the same day. This 
may have been due to the difficulty in scheduling an appointment around 
STETS activities. 

Every month during follov-up fielding, interviewers were sent a 
list of those sample members who were eligible to be interviewed that 
month. While they were instructed to interview the sample member as near 
as possible to the time of the month which corresponded to the random 
assignment date, this was not possible with precision. Therefore, in some 
cases, the follow-up interview was conducted up to 2 weeks before the date 
which marked the end of the Xth month after random assignment (where X is 
6, 15, or 22 months, depending upon the follow-up wave). These cases are 
not included as negative values in calculating the mean; rather, the 
absolute value of the difference in the interview date and the date 
equivalent to X months after random assignment was calculated. This is the 
number of days by which the intei/iew date varied from the date marking 6, 
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TABLE B.5 

TIME BETWEEN ASSIGNMENT AND SAMPLE MEMBER INTERVIEW COMPLETION 

AND LENGTH OF INTERVIEW BY 

INTERVIEW WAVE AND RESEARCH STATUS 



Mean 



Mean Time Between Length of 
Assignment and Completion* Interview 
(Days) (Minutes) 



Baseline Interview 






Experimental 


33.4 


32.7 


Control 


30.1 


30.8 


Total 


31.8 


31.8 


6-ltouth Intetvlev 






Experimental 


14.4 


24.5 


Control 


14.5 


25.4 


Total 


14.4 


24.9 


IS-ilonth Intervlev 






Experimental 


10.3 


24.2 


Control 


9.8 


24.0 


Total 


10.1 


24.1 


22--Month Interview 






Experimental 


11.9 


22.2 


Control 


13.3 


23.0 


Total 


12.6 


22.6 


Number in Sample 






Baseline 


455 


423 


6-Month 


288 


288 


15-Month 


415 


415 


22-Month 


403 


403 



^The number of days between the date of random assignment at baseline and the 
equivalent date in the Xth month after random assignment, where X « 6, 15, or 
22 months depending upon the follow-nip Interview wave. Follow-up interview 
assignments were made at the beginning of the Xth month after random 
assignment, and interviewers were instructed to complete the assignments by 
the end of the month. Because of this, some follow-up interviews were 
conducted before the equivalent date. For that reason, the absolute value of 
the difference in assignment or equivalent data and the interview date Is 
presented in this table. 



B.28 



ERLC 



296 



15) or 22 months after random assignment. At the follow-up waves, the 
number of days between assignment and completion was much shorter than at 
baseline, about two weeks or less on average. There were only small 
differences by research status, and no consistent pattern or trend emerged 
across the follow**up waves. 

The baseline interview averaged about 32 minutes, although 
interviev^ers often spent additional time, apart from the interview Itself, 
to introduce themselves, establish rapport, and explain the study. The 
follow-nip interviews averaged between 23 and 25 minutes depending upon the 
wave. The follow-up instrument did not contain the baseline questions on 
parental background. This factor, plus a greater familiarity with the 
interview process and the questions on the parts of the respondents and the 
interviewers, accounts for the shorter follow-up interview administration 
time. 

Table B.6 presents information on the interview setting. Most 
sample member interviews were conducted in the respondent's home, and 
generally no one else was present during the interview. When someone else 
was present, it was usually a parent or guardian, and, in the vast majority 
of cases, the primairy respondent's answers did not appear to be influenced 
by the presence of others. These patterns apply to all interview waves. 

Table B.7 reports several observations from interviewers pertaining 
to the sample members' orientation toward the interview. Interviewers 
found that the respondents were generally attentive to the interview, 
cooperative, and self-confident. These patterns did not change over the 
course of the study. However, interviewers did believe that the 
respondents' answers were more reliable in later interviewing waves. At 
baseline, over 88 percent of respondents were described as very reliable or 
reliable on most items; by the 22-month follow-up, 94 percent were reported 
equally reliable. This pattern may have been due to interviewers' 
increased appreciation of the abilities of the respondents, to respondents' 
greater familiarity with the questions or to greater knowledge of the 
issues addressed in the interview, or both. 
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TABLE B.6 



LOCATION OF SAMPLE MEMBER INTERVIEW AND PRESENCE OF OTHERS 
DURING INTERVIEW BY INTERVIEW WAVE 
(Percent) 



Baseline 6-month 15-Month 22^onth 
Interview Interview Interview Interview 

Location of Interview^ 



Sample member's home 


60.7 


91.0 


89.9 


90.1 


Home of friend or relative 


1.3 


1.4 


2.4 


3.5 


Agency office 


0.4 


1.0 


2.4 


2.0 


Elsewhere'' 


2.4 


6.6 


5.3 


4.5 


Unknown 


35.2 


0 


0 


0 



Others Present Durlpg 
Interviev^ 



No one 


80.4 


86.8 


83.9 


85.6 


Parent /guardian 


13.2 


8.7 


7.2 


7.2 


Counselor 


0 


0.3 


0.2 


0.3 


Roommate/ friend/spouse 


1.3 


1.4 


2.7 


2.2 


Other 


7.3 


3.5 


8.7 


7.0 



Effect of Presence of Others' 



No others present 


80.4 


86.8 


83.9 


85.6 


Sample member's answers were 










influenced 


3.5 


2.1 


2.2 


2.0 


Sample member's answers were 










not influenced 


16.0 


11.1 


14.0 


12.4 



Number In Sample 455 288 415 403 



The baseline pilot Instrument did not ask Interviewers to record this 
Information. 

^Includes public places such as libraries or restaurants. 

More than one type of person could have been coded as present. 

Based on Interviewer judgment. 
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TABLE B.7 



INTERVIEWER OBSERVATIONS ON SAMPLE MEMBERS 
BY INTERVIEW WAVE 
(Percent) 



Observed Characteristics 
of Sample Members 



Baseline 
Interview 



6-month 
Interview 



15-Month 
Interview 



22-Month 



Attentiveness Duripg 

!•• .Mentally alert, attentive 

2... 

3 • • • 

4... 

5. • .Inattentive 
Not answered 

Gooperativeness During 
Intervlev^ 

1. • .Cooperative 

2... 

3 . . . 

4... 

5. . .Uncooperative 
Not answered 

Self-Confidence Durii^ 
Interview^ 

1... Self-confident 

1... 

3... 

4... 

5. • .Insecure 
Not answered 

Keliability of Respowes^ 

Very reliable 
Reliable on most items 
Reliable on some items 
Very unreliable 
Not answered 



67.7 


73.6 


61.2 


63.3 


22.9 


19.8 


26.0 


27.8 


7.3 


5.2 


9.9 


7.4 


1.8 


1.4 


2.2 


1.2 


0.4 


0 


0.7 


0.3 


0 


0 


0 


0 


86.4 


82.6 


75.2 


76.2 


9.7 


12.5 


15.9 


18.6 


2.6 


4.2 


6.7 


4.5 


1.0 


0.7 


1.9 


0.5 


0 


0 


0.2 


0.3 


0.2 


0 


0 


0 


34.1 


46.2 


35.2 


41.9 


43.1 


37.9 


39.0 


37.5 


17.8 


11.5 


17.3 


1' .6 


4.0 


3.5 


7.2 


4.0 


0.7 


1.0 


1.2 


2.0 


0.4 


0 


0 


0 


49.7 


63.5 


69.2 


73.2 


39.1 


26.4 


22.2 


20.8 


9.0 


8.3 


7.0 


3.2 


2.2 


1.7 


1.7 


2.5 


0 


0 


0 


0.3 



Number in Sample 455 288 415 403 



Only the two extreme points of the scale were labelled in the interview. 

^Data from the primary respondent interviews which were judged "very 
unreliable" or "reliable on some items" and from those with impaired speech 
were replaced with proxy respondent interview data if available. 
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2. Proxy Respoivlents 

During the course of the interview with the sample member, 
interviewers used a number of predetermined checkpoints in the instrument 
to determine whether a proxy respondent was required, based on missing or 
inconsistent data on key interview items. The results of these 
determinations are shown in Table B.8. During the baseline pilot phase, 
proxy interviews were attempted for all respondents; therefore, the pilot 
cases are not included in the baseline column. Pilot study results had 
suggested that about 30 percent of the primary respondent interviews would 
contain missing or inconsistent data, and therefore would require a proxy 
interview. In general, a lower percentage of cases actually required a 
proxy respondent, from 25 percent during the post-pilot baseline to only 13 
percent by the 22-inonth follow-up wave. Moderate variation occurred by 
site, particularly at baseline, where the percentages of cases which 
required a proxy interview ranged from about 15 percent in Los Angeles to 
44 percent in Tucson. This variation decreased over time to an 11.1 
percentage point different at month 22 between New York and St. Paul. This 
secular trend is consistent with the pattern in interviewers* subjective 
judgments about the reliability of sample member data. While, overall, the 
control group sample was more often identified as requiring proxy 
rrfspondents, this was not true in all sites. Generally, however, the 
differences between the experimental and control groups in the percentage 
of cases which required proxy respondents narrowed after the 6-month 
interview. By the 22-month wave, there was only a 4 percentage point 
difference on average. 

Tables B.9 through B.12 present the final status and response rate 
results for proxy respondents in each interview wave. In all waves, 
virtually all of the assigned proxy respondents completed interviews. This 
held true in all sites and for both the experimental and the control group. 

In general, proxy interviews were completed soon after the sample 
member interview was completed, as shown in Table B.13. The average 
elapsed time between the interviews was approximately 2 days at all 
waves. As reported in Table B.14, most proxy interviews were conducted in 
person. Since interviewers were required to conduct all proxy interviews 

B.32 



ERLC 



3u0 



TABLE B.8 



WHETHER PROXY RESPONDENT REQUIRED, 
BY INTERVIEW WAVE, 
SITE, AND RESEARCH STATUS 
(Percent) 





Baseline 


B^Tiionth 


15-Month 


22-Month 

im Lk\J 11 L>ll 




Interview^ 


Interview 


Interview 


Interview 


doclimatl 










Experiment a}. 


17.1 


13.8 


17.1 


15.0 


Control 


25.0 


17.2 


1 S 9 


1 S A 

1 J. H 


Total 


21.1 


15.5 


16. 5 


1 S 0 


Los Ajqgeles 










Experimental 


13.5 


3.3 


10.6 


8.3 


Control 


15.9 


18 S 




1 51 A 
1 O. H 


Total 


14.6 


10. *= 


1 S 1 


X ^ . O 


New York 










Experimental 


36.8 


20.0 


12.0 


6.3 


Control 


23.8 


29.7 


14.6 


1 3.0 


Total 


30.0 


23.3 


13.3 


9.6 


St. Paul 










Experimental 


40.7 


27.8 


30.8 


19.2 


Control 


25.9 


27.8 


18.5 


22.2 


Total 


33.3 


27.8 


24.5 


20.7 


Tucson 










Experimental 


27.8 


18.2 


14.3 


12.8 


Control 


61.1 


22.6 


11.4 


11.^ 


Total 


44.4 


20.3 


12.9 


12.1 


Tbtal 










Experimental 


23.8 


15.1 


15.5 


11.5 


Control 


26.7 


23.2 


15.8 


15.5 


Total 


25.3 


19.1 


15.7 


13.4 


Number in Sample 


297 


288 


415 


403 



NOTE: Whether a proxy respondent was required was determined by the 

interviewer on the basis of the missing or inconsistent responses in 
the pririary respondent interview. The completion of proxy interviews 
did not necessarily mean that proxy data replaced primary respondent 
interview data for analysis. The use of proxy data for analyses was 
determined by explicit rules discussed in Chapter III. 

^Does not include baseline pilot data; during pilot study, proxy interviews 
were attempted for all respondents. 



TABLE B.9 



PROXY BASELINE INTERVIEW 
FlWi STATUS AND RESPONSE RATE BY SITE AND RESEARCH STATUS 

(Percent) 



Final Status 



Copiplete/ Unable to 
Total Partial Proxy Non-English- Locate or Response 
Assigned Complete Refused Speaking Contact Rate^ 



Cincinnati 

Experimental 

Control 

Total 



18 
18 
36 



100.0 
97.2 



0 
0 
0 



0 
0 
0 



5.5 

0 

2.8 



100.0 
97.2 



Los Angeles 

Experimental 

Control 

Total 

New Yoric 

Experimental 

Control 

Total 

St. Paul 

Experimental 

Control 

Total 

Tucson 

Experimental 

Control 

Total 



7 100.0 

7 100.0 

n 100.0 

100.0 

39 97A 

m 98.8 

10 100.0 

7 100.0 

17 100.0 

37 100.0 

42 100.0 

79 100.0 



0 0 

0 0 

0 0 

0 0 

0 2.6 

0 1.2 

0 0 

0 0 

0 0 

0 0 

0 0 

0 0 



0 100.0 

0 100.0 

0 100.0 

0 100.0 

0 97A 
0 98.8 

0 100.0 

0 100.0 

0 100.0 

0 100.0 

0 100.0 

0 100.0 



Total 

Experimental 117 99.1 0 0 0.9 99.1 

Control 113 99.1 0 0.9 0 99.1 

Total 230 99.1 0 OA OA 99.1 



Defined as the number of completed Interviews divided by the total number assigned. There were no proxy 
respondents Ineligible for the survey because of death, Incarceration, or relocation out of the study 
area. 
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TABLE B.10 



PROXY 6.M0NTH INTERVIEW FINAL STATUS AND 
RESPONSE RATE BY SITE AND RESEARCH STATUS 
(Percent) 



Final Status 







Comolete/ 






lin ;ih 1 A f A 






Total 


Partial 


Proxy 


Nnn-Fnnll ^h- 




Response 




Assigned 


Complete 


Refused 


Speaking 


Contact 


Rate^ 


















T 






u 


u 


100.0 




s 




u 


u 


u 


100.0 


Total 


9 


100.0 


0 


0 


0 


100.0 


















1 


100.0 




n 
u 


u 


lUU.U 


Control 


3 


100 0 




n 
u 


u 


1UU.U 


Total 


6 


100.0 


0 


0 


0 


100.0 


New York 














Experimental 


6 


100.0 


0 


0 


0 


100 0 


Control 


11 


90.9 


0 


9.1 


0 


90 9 


Total 


17 


%.1 


0 


5.9 


0 




St« Paul 














Experimental 


5 


100.0 


0 


0 


0 


100 0 


Control 


5 


100.0 


0 


0 


0 


100 0 


Total 


10 


100.0 


0 


0 


0 


100.0 


Tucson 














Experimental 


6 


100.0 


0 


0 


0 


100.0 


Control 


7 


100.0 


0 


0 


0 


100.0 


Total 


13 


100.0 


0 


0 


0 


100.0 


Total 














Experimental 


22 


100.0 


0 


0 


0 


100.0 


Control 


33 


97.0 


0 


3.0 


0 


97.0 


Total 


55 


98.2 


0 


1.8 


0 


98.2 



defined as the nutnber of completed Interviews divided by the total number assigned. There were no proxy 
respondents Ineligible for the survey because of death, Incarceration, or relocation out of the study 
area. 



TABLE B.11 



PROXY 15-MONTH INTERVIEW FINAL STATUS AND 
RESPONSE RATE BY SITE AND RESEARCH STATUS 
(Percent) 



Final Status 







Complete/ 






Unable to 






Total 


Partial 


Proxy 


Non-English- 


Locate or 


Response 




Assigned 


Complete 


Refused 


Speaking 


Contact 


Rate^ 


Cincinnati 














Experimental 


7 


100.0 


0 


0 


0 


100.0 


Control 


7 


100.0 


0 


0 


0 


100.0 


Total 




100.0 


0 


0 


0 


100.0 


Los Angeles 














Experimental 


5 


100.0 


0 


0 


0 


100.0 


Control 


8 


100.0 


0 


0 


0 


100.0 


Total 


13 


100.0 


0 


0 


0 


100.0 


New York 














Experimental 


6 


100.0 


0 


0 


0 


100.0 


Control 


7 


100.0 


0 


0 


0 


100.0 


Total 


13 


100.0 


0 


0 


0 


100.0 


St. Paul 














E .perlmental 


8 


100.0 


0 


0 


0 


100.0 


Control 


5 


100.0 


0 


0 


0 


100.0 


Total 


13 


100.0 


0 


0 


0 


100.0 


Tucson 














Experimental 


7 


100.0 


0 


0 


0 


100.0 


Control 


5 


100.0 


0 


0 


0 


100.0 


Total 


12 


100.0 


0 


0 


0 


100.0 


Total 














Experimental 


33 


100.0 


0 


0 


0 


100.0 


Control 


32 


100.0 


0 


0 


0 


100.0 


Total 


63 


100.0 


0 


0 


0 


100.0 



defined as the iiumber of completed Interviews divided by the total nurrtber assigned. There were no proxy 
respondents Ineligible for the survey because of death, Incarceration, or relocation out of the study 
area. 
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TABLE B.12 



PROXY 22-MONTH INTERVIEW FINAL STATUS AND 
RESPONSE RATE BY SITE AND RESEARCH STATUS 
(Percent) 



Final Status 







C'^plete/ 






Unable to 






Total 


Partial 


Proxy 


Non-English- 


Locate or 


Response 




Assigned 


Complete 


Refused 


Speaking 


Contact 


Rate^ 


Cindmati 














Experimental 


6 


100.0 


0 


0 


0 


100.0 


Control 


6 


100.0 


0 


0 


0 


100.0 


1 Ot^ 


12 


100,0 


0 


0 


0 


100.0 


Los Angeles 














Experimental 




100.0 


0 


0 


0 


100.0 


Control 


7 


100.0 


c 


0 


0 


100.0 


Total 


11 


inn n 


u 


0 


0 


100.0 


New Yorfc 














Experimental 


3 


lOO.O 


0 


0 


0 


100.0 


Control 


6 


100.0 


0 


0 


0 


100.0 




O 

if 


inn n 


0 


0 


0 


100.0 


St. Paul 














Experimental 


6 


100.0 


0 


0 


0 


100.0 


Control 


5 


100.0 


0 


0 


0 


100.0 


Total 


11 


100.0 


0 


0 


0 


100.0 


Tucson 














Experimental 


6 


100.0 


0 


0 


0 


100.0 


Control 


5 


100.0 


0 


0 


0 


100.0 


Total 


11 


100.0 


0 


0 


0 


100.0 


Total 














Experimental 


25 


100.0 


0 


0 


0 


100.0 


Control 


29 


100.0 


0 


0 


0 


100.0 


Total 


5^^ 


100.0 


0 


0 


0 


100.0 



defined as the nunter of completed Interviews divided by the total nufrtber assigned. There were no 
proxy respondents Ineligible for the survey because of death, Incarceration, or relocation out of the 
study area. 
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AVERAGE TIME BETWEEN SAMPLE MEMBER AND PROXY INTERVIEWS, 
BY 1 "FaVIEW wave and RESEARCH STATUS 
(Days) 





Experimental 


Control 






Group 


Group 


Total 


Baseline Interview 


1.6 


1.8 


1.7 


6-Month Interview 


0.9 


3.6 


2.5 


15-Month Interview 


2.4 


1.8 


2.1 


22-Month Interview 


1.9 


2.3 


2.2 


Number in Sample 








Baseline 


117 


114 


231 


Month 6 


22 


33 


55 


Month 15 


33 


32 


65 


Month 22 


24 


30 


54 



NOTE: Average is defined as arithmetic mean# Only those cases in which 
the interviewer determined that a proxy respondent interview was 
required and it was completed are included in this table. Whether a 
proxy respondent was required was determined by the interviewer on 
the basis of missing or inconsistent responses in the primary 
respondent interview. The completion of proxy interviews did not 
necessarily mean that proxy data replaced primary respondent 
interview data for analysis. Cases in which the proxy respondent 
interview waB administered immediately after, or on the same day as, 
the primary respondent interview are coded as "zero" elapsed days. 
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TABLE B,14 



MODE OF PROXY INTERVIEW ADMINISTRATION 
AND TYPE OF RESPONDENT BY INTERVIEW WAVE 
(Percent) 





Baseline 


o^Montn 


1 J— Month 


22-Month 




inLerview 


Interview 


Interview 


Interview 












In person 

Bv t*p1 pnVinnP ^ ' 

Vjr U ItV^ lie 


93.4 


87.0 


87.7 


81.5 




1 n 
1 J • u 




1 fl <: 
io. 3 












Parent 


74.2 


72.7 


63.1 


61.1 


Foster parent 


4.4 


9.1 


6.2 


5.5 


Legal guardian 


0 


0 


1.5 


1.9 


Other relative 


6.1 


5.5 


0 


3.7 


Roommate or friend 


0.9 


0 


0 


0 


Residential house parent 


4.8 


1.8 


7.7 


9.3 


Agency staff member 


9.2 


10.9 


21.5 


13.0 


Other 


0.4 


0 


0 


5.5 


Number in Sample 


229 


55 


65 


54 
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in person during the baseline pilot study, the table includes only those 
baseline interviews completed after the pilot study. Even so, proxy 
respondents were overwhelming interviewed in person throughout the study, 
due largely to the fact that very often the proxy interview immediately 
followed the in-person interview with t.^a sample member. 

Table B.IA also reports the distribution of proxy respondents by 
relationship to the primary respondent. In any given interview wave, most 
proxy respondents were parents of the sample members. However, in later 
(15- and 22-month) waves, over 20 percent of the proxy respondents were 
agency staff or residential counselors. 

3. Missing Data 

The amount of missing data (don't know responses or unusable 
responses) is a critical measure of the quality of response in self- 
reported interviews. Previous research with mentally retarded populations 
and our own experience in the baseline pilot study led us to expect missing 
data from sample members on certain items that were central to the 
evaluation. These items were generally those that pertained to the amount 
of money received from earnings and benefit programs, as well as to the 
identity of the specific programs from which cash and other benefits were 
received. This expectation was confirmed by the percentage of missing data 
on key items, as reported in Tables B.I5 through B. 18. The first column of 
percentages in these tables is the percentage of responses recorded as 
"don't know" or uncodable. (Refusals to provide information are not 
treated as missing data in these tables.) For the sample members 
themselves, the items on which 5 percent or more of the data were missing 
were those pertaining to transfer program benefit receipt and the amount of 
earnings. There was very little missing data on activities or details of 
living arrangements. These patterns held true in all waves, with the 
amount of SSI or SSDI benefits consistently having the highest level of 
missing data. This is not surprising, given the fact that, in many cases, 
these benefits were handled by someone else on behalf of the sample member. 

What is more surprising, perhaps, is the fact that proxy 
respondents were not necessarily more knowledgeable. Particularly in terms 
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TABLE B.15 



BASELINE INTERVIEW 
MISSING DATA BY SOURCE 
(Percent) 



Category 




Sample Member 


Proxy 


Combined 


of Variable 


Variable 


Interviews 


Interviews 


Data^ 


Labor Market 


Employment Status 


0.2 


1.7 


0.4 


Outcomes 


Hours Worked per Week'^ 


2.0 


10.9 


1.5 




Weekly Earnings'^ 


5.5 


28.4 


4.6 


Training and 


In Training" 


3.7 


6.5 


2.9 


Schooling 


In School'' 


0 


0 


0.2 


Public Transfer 


Receipt of SSI or SSDI'^ 


11.2 


3.5 


1.1 


Dependence 


Receipt of Other Cash 










Transfers'^ 


5.7 


1.7 


0.7 




Receipt of Food Stamps'^ 


4.2 


2.6 


0.7 




Receipt of Medicare or 










Medicaid'^ 


5.9 


3.5 


0.9 




Amount of SSI or SSDI^ 


15.8 


5.7 


1.3 




Amount of Other Cash 








Transfers^ 


8.8 


1.7 


2.4 


Independence and 


Financial Management Activities 






Life Style 


Pays for purchases by 


self 0.2 


0.4 


0.4 




Banks by self 


0.7 


0.4 


0.4 




Pays bills by self 


1.3 


2.2 


1.8 




Living Arrangements 


0.7 


0.4 


0.2 


Number in Sample 




455 


229 


455 



NOTE: Missing data refers only to "don't know" responses or other responses 
indicating an inability to provide accurate information (e.g., vague, 
uncodable, or unintelligible responses). Refusals to provide the requested 
information are not included as missing data. 



^Data are combined after identifying the items on which the sample member data were 
missing and after replacing them with proxy data if available. 

"don't know" response by the sample member was an indicator to use proxy data for 
the entire work/school/training module. 

"don't know" response by the sample member was an indicator to use proxy data for 
this variable, except for certain special circumstances. 

^Based on the pilot study results, "don't know" responses by sample members to 
questions about the receipt of transfers were treated as no receipt for the 
purposes of analysis. However, "don't know" responses on these items are included 
as missing in this table. 

^A "don't know" response to any question on the amount of transfer received was an 
indicator to use proxy data ror all information on the receipt of transfers. 
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TABLE B.16 



6-MONTH FOLLOW-UP INTERVIEW 
MISSING DATA .iY SOURCE 
(Percent) 



Category 




Sample Member 


Proxy 


Combined 


of Variable 


Variable 


Interviews 


Interviews 


Data* 


Labor Market 


Employment Status*^ 


0 


1.8 


0 


Outcomes 


Hours Worked per Week"- 


1.4 


12.7 


0.7 




Weekly Earnings'^ 


4.5 


16.4 


1.7 


Training and 


In Training" 


0.3 


3.6 


0.3 


Schooling 


In School'' 


0 


0 


0 


Public Transfer 


Receipt of SSI or SSUI^ 


9.4 


3.6 


0.3 


Dependence 


Receipt of Other Cash 










Transfers*^ 


6.9 


0 


0 




Receipt of Food Stamps'^ 


4.9 


0 


0 




Receipt of Medicare or 










Medicaid 


2.4 


0 


0 




Amount of SSI or SSDI^ 


10.8 


3.6 


0.3 




Amount of Other Cash 










Transfers^ 


4.2 


1.8 


1.4 


Independence and 


Financial Management Activiti>. ■• 






Life Style 


Pays for purchases by self 0 


1.8 


0 




Banks by self 


1.0 


0 


1.0 




Pays bills by self 


0.7 


0 


0.7 




Living Arrangements 


0.3 


0 


0 


Number in Sample 




288 


55 


288 



NOTE: Missing data refers only to "don't know" responses or other responses 
indicating an inability to provide accurate information (e.g., vague, 
uncodable, or unintelligible responses). Refusals to provide the requested 
information are not included as missing data# 

^Data were combined after identifying the items on which the sample member data were 
missing and after replacing them with proxy data if available, 

"don't know" response by the sample member was an indicator to use proxy data for 
the entire work/school/training module* 

"don't know" response by the sample member was an indicator to use proxy data for 
this variable, except for certain special circumstances* 

^Based on the pilot study results, "don't know" responses by sample members to 
questions about the receipt of transfers were treated as no receipt for the 
purposes of analysis. However, "don't know" responses on these items are included 
as missing in this table. 

^A "don't know" response to any question on the amount of transfer received was an 
indicator to use proxy data for all information on the receipt of transfers. 
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TABLE B.17 



15-MONTH FOLLOW-UP INTERVIEW 
MISSING DATA BY SOURCE 
(Percent) 



Category 




Sample Member 


Proxy 


Combined 


ot variaDie 


Variable 


Interviews 


Interviews 


Data* 


Labor Nartcec 


Employment Status^ 


0.2 


1.5 


0 


Outcomes 


IT ^ TT 1. J _ TT if* 

Hours worked per Week^ 


2.2 


6.1 


0.5 




Weekly Earnings'^ 


6.5 


21.5 


3.1 


Training and 


■ 

In Training" 


0.2 


3.1 


0.2 


Schooling 


In School^ 


0 


0 


0 


Public Transfer 


Receipt of SSI or SSDI'^ 


4.3 


1.5 


0 


Dependence 


Receipt of Other Cash 










Transfers'^ 


4.6 


1.5 


0 




Receipt of Food Stamps'^ 


2.7 


0 


0 




Receipt of Medicare or 










Medicaid'^ 


3.4 


3.1 


0.2 




Amount of SSI or SSDI^ 


9.4 


10,8 


1.7 




Amount of Other Cash 










Transfers^ 


3.6 


1.5 


0.5 


Independence and 


Financial Management Activities 






Life Style 


Pays for purchases by 


self 0 


0 


0 




Banks by self 


1.0 


1.5 


0.7 




Pays bills by self 


0.7 


1.5 


0.7 




Living Arrangements 


1.0 


0 


0.2 


Number In Sample 




415 


65 


415 



NOTE: Missing data refers only to "don*t know" responses or other responses 
Indicating an Inability to provide accurate Information (e.g., vague, 
uncodable, or unintelligible responses). Refusals to prOv^lde the requested 
Information are not Included as missing data. 



^Data were combined after Identifying the Items on which the sample member data were 
missing and after replacing them with proxy data If available. 

"don^t know" response by the sample member was an Indicator to use proxy data for 
the entire work/school/tralnlng module. 

"don^t know" response by the sample member was an Indicator to use proxy data for 
this variable, except for certain special circumstances. 

^Based on the pilot study results, "don*t know" responses by sample members to 
questions about the receipt of transfers were treated as no receipt for the 
purposes of analysis. However, "don*t know" responses on these Items are Included 
as missing in this table. 

^A "don*t know" response to any question on the amount of transfer received was an 
indicator to use proxy data for all information on the receipt of transfers. 
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TABLE B.18 



22-MONTH FOLLOW-UP INTERVIEW 
MISSING DATA BY SOURCE 
(Percent) 



Category 
of Variable 


Sample Member 
Variable Interviews 


Proxy 
Interviews 


Combined 
Data^ 


Labor Market 


Emplojrment Status^ 


0.2 


0 


0 


Outcomes 


Hours Worked per Week^ 


2.0 


5.6 


0.3 




Weekly Earnings^ 


4.5 


5.6 


0.7 


Training and 


In Training*^ 


0*7 


1.9 


0.5 


Schooling 


In School^ 


0 


0 


0 


Public Transfer 


Receipt of SSI or SSDI^ 


4.0 


7.4 


1.0 


Dependence 


Receipt of Other Cash 










Transfers^ 


3.0 


1.9 


0.3 




Receipt of Food Stamps^ 


1.7 


0 


0 




Receipt of Medicare or 










Medicaid^ 


4.0 


1.9 


0.3 




Amount of SSI or SSDI^ 


8.9 


5.6 


1.0 




Auiount of Other Cash 










Transfers^ 


2.2 


0 


0.5 


Independence and 


Financial Management Activities 








Life :^tyle 


Pays for purchases by self 


0.2 


1.9 


0 




Banks by self 


0.5 


0 


0.3 




Pays bills by self 


1.0 


1.9 


0.7 




Living Arrangements 


0.5 


0 


0 


Number In Sample 




403 


54 


403 



NOTE: Missing data refers only to "don*t know" responses or other responses 
Indicating an Inability to provide accurate Information (e.g., vague, 
uncodable, or unintelligible responses). Refusals to provide the requested 
Information are not Included as missing data. 

^Data are combined after Identifying the Items on the which sample member data were 
missing and after replacing them with proxy data If available. 

**A "don*t know" response by the sample member was an Indicator to use proxy data for 
the entire work/school/ training module. 

^A "don*t know" response by the sample member was an Indicator to use proxy data for 
this variable, except for certain special circumstances. 

Based on the pilot study results, "don*t know" responses by sample members to 
questions about the receipt of transfers were treated as no receipt for the 
purposes of analysis. However, "don*t know" responses on these Items are Included 
as missing In this table. 

®A "don*t know" response to any question on the amount of transfer received was an 
indicator to use proxy data for all Information on the receipt of transfers. 
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of hours and earnings from jobs, proxy respondents consistently reported 
more missing data than did the sample members who had been identified as 
being unable to provide complete and consistent information on their own. 

In fact, not all of the proxy data available proved necessary in 
constructing the analysis file. In some cases, the proxy data were no 
better than the sample member data for missing or inconsistent items; in 
other cases, sample member data were accurate and complete for most items 
and needed supplementation from the proxy interview only for a few items; 
in still others, the entire case had to be based on proxy data. As 
summarized in Table III. 7 in Chapter III, only about 6 percent of the 
sample member interviews were replaced entirely with data from proxy 
respondents. Substantial proportions (up to 16 percent) used proxy data 
for the transfer benefit module, although this percentage declined (to 7 
percent) over the interviewing period. Once the two sources— sample member 
and proxy respondent data— vere merged, very little missing data remained, 
as is shown in the last column of Tables B.15 through B.18. 

D. SUMMARY: METHODOLOGICAL CONTRIBUTIONS 

The rigorous evaluation of the STETS demonstration makes an 
important contribution to research and policy-making in the area of 
programs for the mentally retarded. The data collection effort associated 
with the evaluation also marks a milestone in research with this 
population. The STETS data collection is probably the largest systematic 
effort to obtain the data necessary for quantitative anplysis directly from 
mentally retarded persons themselves. Certainly, there were limitations on 
the data which seemed feasible to collect in a self-reported interview— in 
particular, data on past experiences— and it was necessary in some cases to 
collect and use data from parents, counselors, and other proxy 



Based on the pilot study results, "don't know" responses by sample 
members to questions about the receipt of transfers were treated as no 
receipt for the purposes of analysis. 
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respondents. However, the great majority of mentally retarded persons in 
our sample provided complete and accurate information on all variables of 
interest. While the same level of success cannot be guaranteed with all 
mentally retarded persons, this study should make self-reported interviews 
a serious option in future research efforts, and provide a nsodel for 
questionnaire design and data collection procedures to maximize the quality 
of self-reported datas 



Accuracy was evaluated in the pilot study through comparisons ot 
sample member responses with those of proxy respondents and information in 
STETS intake records; it was evaluated during the full study through 
comparisons of self-reports of activities and types of services received 
with program records and service agency interviews. 
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APPENDIX C 

BENEFIT-COST ANALYSIS METHODS AND RESULTS 
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This appendix describes the estimates and valuation assumptions 
that underlie the benefit-cost findings presented in Chapter VIII. Those 
findings reflect both numerous assumptions about the effects of STETS on 
participants' behavior and the appropriate values of those effects. While 
we feel that these assumptions are reasonable, an inherent uncertainty in 
the analysis remains. The information presented in this appendix on the 
rationale and use of these underlying assumptions and on the sensitivity of 
the results to changes in these assumptions provides a basis for judging 
this inherent uncertainty and the validity of the overall findings. 

Throughout the analysis, we emphasized that the benefit-cost 
analysis represents a method for drawing together information on the 
diverse impacts and costs of STETS. It compares the estimated values of 
benefits and costs in order to help form judgments about the effectiveness 
of the program overall. Thus, the individual dollar estimates of benefits 
and costs are intended to suggest the order-of -magnitude of these values, 
rather than to represent precise measures of value, implying that attention 
should be paid to the pattern of estimates obtained in analysis, and not to 
the specific dollar estimates. 

Our review of the underlying assumptions is organized into the 
following six sections: 

A. General valuation procedures 

B. Program cost estimates 

C. The value of Phase 1 and Phase 2 output 

D. Impact estimates for the benefit-cost analysis 

E. Estimated values for impacts 

F. Sensitivity tests 

The last section examines how the overall findings would be affected by 
changes in the valuation assumptions and procedures. It therefore provides 
a method for summarizing and assessing the level of uncertainty inherent in 
the analysis. 
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A. GENERAL VALUATION PROCEDURES 

Chapter VIII presented our basic approach to the benefit-cost 
analysis. The approach is similar to the approaches used in evaluations of 
the national Supported Work demonstration (Kemper, Long, and Thornton, 1981 
and 1984) and Job Corps (Thornton, Long, and Mallar, 1982). It also uses 
basic benefit-cost accounting procedures, wliich are described in Gramlich 
(1981) and other texts on benefit-cost analysis. 

The analysis attempts to estimate the effect of STETS on the use of 
resources and the distribution of resources between the participant group 
and the nonparticipant group (i.e., all other persons in society). It does 
so by estimating the market value of any use, creation, or savings of 
resources. For example , increased output produced by participants when 
they leave STETS is valued on the basis of what employers pay those 
participants to produce the output. Similarly, when participants reduce 
their use of programs other than STETS, we estimate the value of the 
resulting resource savings on the basis of what it would have cost the 
affected program to provide the foregone services. This focus on resource 
use is similar to the approach used to calculate gross national product. 

Of course, the focus on resource costs fails to captute a number of 
potentially important intangible benefits that are listed in the benefit- 
cost accounting framework (Table VIII. 1). For example, the preferences of 
participants to be employed productively or to be more self-sufficient are 
not captured by the resource-cost approach. Similarly, the altruistic 
attitudes of nonparticipants are excluded. However, because these and the 
othey intangible benefits and costs represent important objectives and 
impacts of STETS, we have made an effort to include them in the analysis, 
even though they are not explicitly valued. Therefore, throughout the 
analysis, it is important to remember that the resource-cost approach 
captures only some of the components of the accounting framework; judgments 
about program effectiveness must reflect all the components and thus must 

consider the dollar value of changes in resource use and the potential 

s 

magnitude of intangible benefits and costs. 
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In using the resource-cost approach, we corrected for the effects 
of inflation on the dollar value of goods and services. The data used in 
the evaluation cover the period from late 1981 to fall 1984. During this 
period, the general price level, as measured by the implicit price deflator 
for gross national product, rose by 21 percent.^ Thus, the actual goods 
and services represented by a dollar changed over the period. To ensure 
that our benefit-cost estimates reflect changes in resource use rather than 
in price levels, we value all benefits and costs in 1982 dollars. 

This base^ period was chosen because it corresponds to the period of 
"steady-state" operations identified by Riccio and Price (1984; 127ff), and 
because the program cost estimates are based on data from this period. By 
using 1982 as the base period, we did not have to adjust the program cost 
data. For other dollar-denominated benefits and costs (for example, 
earnings and transfer payments), we multiplied the amounts reported in the 
interviews by the percentage change in the implicit price deflator between 
1982 and the interview date. The other benefits and costs, such as months 
in a sheltered workshop or school, were valued by multiplying the change in 
months of use by a "shadow price" that indicated the monthly cost of using 
that program in 1982 dollars. Readers who are interested in the current 
value of the benefits and costs disc.issed in this report can estimate that 
value by multiplying our estimates by the change in the implicit price 
deflator between 1982 and the current period. For example, to translate 
the 1982 dollar estimates into first-quarter 1985 dollars, one would have 
to increase the figures presented in the benefit-cost analysis by 10 
percent, which is the approximate percentage change in the implicit price 
deflator between 1982 and the first quarter of 1985. 

An additional adjustment beyond the inflation adjustments is 
necessary before benefits and costs that occur at different times can be 
compared. This adjustment discounts values over time to thei* equivalent 
present value. Discounting is necessary because a benefit or cost 



1 

The implicit price deflator for gross national product is used, 
rather than other indicators of price levels, because it is more broadly 
based. The implicit price deflator essentially reflects prices for all 
commodities used in the economy rather than prices for a specific set of 
goods, as is the case with price indexes such as the consumer price index. 
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(measured as a given araount of dollars) achieved this year is worth more 
than one achieved, say, ten years from now, even after inflation has been 
taken into account. Consider a result that increased participant 
earnings* These increased earnings, if they occurred this year, could be 
reinvested and could earn a rate of return over the next ten years. Thus, 
over a ten-year period, the value of the increase in earnings will equal 
the initial increase plus the return on investment over the next ten 
years. This value will clearly exceed the value of the same increase if it 
occurred ten years from now. 

Because of these differec^ces in value, the analysis adjusts all 
estimated benefits and costs to equivalent values by discounting those that 
occur in the future by a factor that reflects the return that could have 
been earned in the interim. The resulting discounted values are termed 
present values.** We use the time of randomization as our base period for 
discounting. Thus, all discounted values presented in the report indicate 
the present value at the time of random assignment (i.e., enrollment in the 
demonstration). 

The appropriate discount rate in evaluating social programs is 
always somewhat controversial because, although the choice of a discount 
rate is very important for the evaluation and is well established 
theoretically, there has never been a completely satisfactory way to 
estimate discount rates. Imperfections in the markets for capital, the 



1 

Suppose that a $1,000 benefit occurs 10 years from now. What 
present value invested at 5 percent return per annum would yield $1,000 ten 
years from now? Call that value PV. PV invested today would earn 5 
percent a year for 10 years, or (1 + .05)^^. Thus, its value 10 years from 
now is PV (1 + .05)1° ^^^j^ $1,000. Divide both sides by (1 + .05) to 
obtain the present value = $1,000 , or $614. This figure is the 

(1 + .05)^° 

present value of a $1,000 benefit that would occur 10 years from now. 
2 

Baumol ( 1968) provides a theoretical foundation for measuring the 
social discount rate. He suggests that it should measure the rate of 
return that the resources used for the public investment would have earned 
otherwise in the private sector. Bradford (1975) suggests using the rate 
at which consumers trade off future for current consumption (the social 
rate-of-time preference). These approaches lead to the same rate if all 
markets are competitive. However, in the presence of markets that are 
characterized by monopoly power, inflation, taxes, and uncertainty, the 
approaches lead to quite different results and are difficult to implement 
empirical ly. 
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existence of risk, uncertainty, and inflation, and the fact that many tax- 
incidence questions are still unresolved have made it impossible to 
determine a single discount rate that is appropriate for evaluating social 
programs. Consequently, the discount rate is typically chosen 

arbitrarily. Most studies of social programs have used rates of between 3 
and 10 percent a year. Our procedure is to assume a middle value, 5 
percent, and then to cest the sensitivity of the findings to this 
assumption by recomputing the values using 3 and 10 percent discount 



rates. 



For employment and training programs such as STETS, the social net 
present value (the basic benefit-cost criterion) will change in an opposite 
direction from a change in the discount rate, because social costs are 
generally incurred during the in-program period (hence, their value is not 
changed much by discounting to the time of enrollment), while the benefits 
accrue over many time periods. Therefore, if a higher rate is used, the 
present value of future benefits will fall, and, because costs are 
essentially unaffected, estimated net present value will decline. 

B. PROGRAM COST ESTIMATES 

As stated in Chapter VIII, program costs consist of three 
elements: operational costs, participant compensation, and central 
administrative costs. In analyzing all three types of costs, we recognize 
that the cost estimates must be consistent with the impact estimates. In 
particular, the costs should reflect the resources required to generate the 
impacts. We want to include the costs of any program activity that has 
affected participant behavior and to exclude the costs of activities that 
did not affect the impacts. 



1 

/,'?-7^N^° percent rate is mandated by the Office of Management and 
Budget (1972) for evaluating government investments. All the discount 
rates are net of inflation; given current inflation rates of 4 percent per 
year, the 3 to 10 percent range in real rates corresponds to nomlmal 
interest rates of 7 percent to 14 percent. 
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What costs, if any, should be excluded depends on (1) the costs of 
the research and evaluation component of STETS and (2) the extraordinary 
costs of program operation due to the fact that STETS was a 
demonstration. The activities associated with these costs are not part of 
the basic STETS model and would not be incurred in an ongoing program. 
Moreover, it was assumed that these activities did not affect the delivery 
of services to participants or the subsequent behavior of participants* 
Consequently, we excluded these costs from the cost estimates presented in 
Chapter VIII. 

This assumption seems quite reasonable in terms of the research 
costs. Most of these costs at the project level pertained to the 
additional outreach and screening efforts necessary to identify individuals 
who were ultimately assigned to the control group, and to the efforts 
necessary to complete the evaluation components of the client and cos t- 
nsonitoring forms. Neither of these activities was likely to have affected 
the observed impacts. 

The assumption is more tenuous in terms of the extraordinary costs 
incurred by the projects because STETS was operated as a demonstration. 
Many of these costs were attributable to start-up activities and to the 
small scale of the demonstration, and it is plausible that a larger ongoing 
program would be able to provide STETS services at a lower average cost. 
The uncertainty of the assumption exists particularly because the small 
scale may have enabled the demonstration programs to provide more intensive 
services. This scenario could have occurred, for example, during the 
start-up and phase-down periods of the demonstration, when the ratio of 
staff to clients was higher than during the steady-state period. If rac*-e 
intensive services were provided, then the extra costs associated with 
those services should be included. 

We have no way of knowing whether more intensive services were 
delivered or what they cost. Even if the services could be identified, it 
would be extremely difficult to separate the costs of any intensive 
services from the normal start-up costs and from the costs due to the 
relatively small scale of operations. Nevertheless, by using an upper- 
bound estimate of costs that includes all operating costs of the 
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demonstration rather than only those costs that were incurred during the 
steady-state period, we can test the sensitivity of our overall findings to 
changing the assumption that the extraordinary demonstration operating 
costs did not affect program impacts. 

We estimated both steady-state and total costs by using data 
provided by MDRC's demonstration accounting system. To estimate steady- 
state average cost per participant, we first subtracted estimated research 
costs from steady-state operating costs. We then divided by the number of 
active months for the period and multiplied the result by the average 
active months per participant for the entire demonstration period/ The 
resulting estimates, which are presented in Table C.l, indicate what it 
would have cost to serve an average STETS participant had all participants 
been served during the steady-state period. Using a similar procedure to 
estimate the costs for the entire demonstration period, we find that the 
average operating costs per participant for the entire demonstration 
(excluding estimated research costs) exceeded the steady-state operating 
costs by 32 percent ($8,221, compared with $6,211). The data also indicate 
that participant compensation per participant was lower over the entire 
demonstration than during the steady-state period. Section F examines how 
these differences affect the overall benefit-cost conclusions. 

In contrast to operational costs and participant compensation, the 
central administrative costs were not estimated on the basis of the 
demonstration experience. As we noted in Chapter VIII, MDRC's dual role as 
monitor and researcher made it impossible to identify the costs of central 



1 

We also could have used enrollment months in making this 
calculation. Doing so would not have altered the estimate of cost per 
participant. While average costs per enrollment month are lower than 
average costs per active month, the average length of participation is 
correspondingly greater when measured in enrollment months rather than in 
active months. We present the costs per enrollment month in Table C.l to 
facilitate comparisons with the net cost per service year estimates in 
Table 7.1 of Riccio and Price (1984); their net cost per service year 
estimate ($8,715), when converted to a monthly amount by dividing by 
twelve, equals operating costs per enrollment month ($552.5) plus 
participant compensation per enrollment month ($282.6) less service project 
revenue per enrollment month ($108.8). 
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TABLE C.I 

PROGRAM CX)ST ESTIMATES FOR THE BENEFIT-COST ANALYSIS 





Steady. State Period 




Entire Demonstration Period 




Operating 

Lose 


Participant 
Compensation 


Service 
Project 
Revenue^ 


Operating 
Cost 


Participant 
Compensation 


Service 
Project 
Revenue^ 


Total for Period 


593,^^98 


281,15^^ 


108,267 


2,496,652 


740,079 


318,518 


Research Costs^ 


^3,733 


— 




161,837 


— 


— 


Net for Period 


549,765 


281 , 1 54 


108,267 


2,334,815 


740,079 


318,518 


Average Per 

Enrollment Month 


552.5 


282.6 


108.8 


783.0 


248.2 


106.8 


Average Per ^ 
Active Month 


666A 


341.0 


131.2 


975.6 


310.0 


133.2 


Net Per Participant^ 


6,211 


3,176 


1,223 


8,221 


2,606 


1,122 



a 



Service project revenue Includes payments received by the projects In exchange for goods 
produced by participants or as compensation for participant labor. For the steady-state 
period, the service project revenue was assumed to be obtained with a three-month lag (see 
Rlcclo and Price, 1984, p. 128, footnote 1). 

b 

Research costs were estimated to be 5 percent of total project-level costs (operating costs 
plus participant compensation); see Rlcclo and Price (1984) p. 128, footnote 3. 

c 

For the steady-state period, there were 995 enrollment months. For the demonstration as a 
whole, there were 2,982 enrollment months. 

d 

For the steady-state period, there were 825 active months. For the demonstration as a whole, 
there were 2,391 active months. 

e 

Cost per participant for the steady-state period Is derived by multiplying the average costs 
per active month by 9.32, the average length of participation In active months for research 
sample members. For the demonstration as a whole, we simply divided total costs by the 
number of participants, 284. These costs are not discounted. 
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administration. We thus estimated these costs on the basis of central 
administrative expenses for existing employment and training programs 
funded by the U.S. Department of Labor (DOL). In fiscal 1982, central 
administrative costs for the Employment and Training Administration at DOL 
were approximately 2 percent of total program expenditures (see the Budget 
of the United States Government, Fiscal 1984, pp. I-Ol to 1-08). If this 
ratio is applied to the steady-state project-level costs (operational costs 
plus participant compensation), the estimated central administrative costs 
would be approximately $20 per active month. This number represents the 
general order-of -magnitude of central administrative costs; the actual 
costs that would be incurred if STETS were implemented on a permanent basis 
may differ according to the level of central monitoring and technical 
assistance provided. 

C. THE VALUE OF PHASE 1 and PHASE 2 OUTPUT 

We estimated the value of output produced by participants on Phase 
1 and Phase 2 jobs by studying the work performed by a randomly selected 
subset of participants. These studies were conducted between September and 
December 1982, a period that corresponds approximately with the steady- 
period. We selected eight active participants from each site and examined 

2 

the output they produced over a two-n/eek period. As part of this 
examination, we interviewed program staff and the participants' work 
supervisors and consulted actual production records. We collected detailed 
production information that included hours worked, wages paid, supervision 
provided, and the amount of work completed. 



The steady-state period differed slightly across sites (Riccio and 
Price, 1984, p. 127), but generally ran from June to October 1982. Our 
value-of-output measurement periods also differed across sites, from late 
September 1982 in Cincinnati to mi<i-December in Los Angeles. 



We studied a total of 40 participants. Of this total, 33 were in 
the research sample, and the others had been enrolled prior to the start of 
randomization. The results for the full sample of 40 and for the subsample 
of 33 are essentially the same. Both sets of results are presented, but we 
focus on the results for the research sample members. 



ERIC 




We made two estimates of the value of output produced. The first 
estimate was the alternative supplier's price of the participant output — 
that is, what an employer would have paid other workers to produce the 
participant output had the participant been unavailable. This estimate 
assumes that employers could obtain additional labor at the wages paid to 
their regular employees, and it ignores any additional costs that may have 
been imposed on the employers who hired STETS participants. The second 
estimate attempts to deduct these additional costs. It estimates the net 
value added by subtracting from the alternative supplier's price the costs 
of extra employer-provided supervision and any reduced output from other 
workers. 

The net value may differ from the alternative supplier's price for 
several reasons. Participants may have required supervision or training 
beyond that which would usually be provided to regular workers. For 
example, one firm which used a participant to clean its offices received 
very little net value, even though the participant performed good work at a 
productive pace. That firm provided considerable training and was 
unwilling to scale back its purchase of alternative cleaning services until 
it could dispense with these services altogether. Alternatively, an 
employer may choose to use the participant labor (often provided at 
subsidized wages) in a way that fails to maximize output. An example of 
this scenario was a firm in which the regular clerical staff "donated" 
their simplest jobs to the participant — photocopying, mailing, and 
answering the telephone during employee breaks. The work performed by this 
participant was both useful and necessary, but reduced the efforts of other 
employees. Finally, the work performed by participants may affect the 
quality of output produced, but the employer may be unable to capture the 
value of this increased quality by increasing prices or output. This was 
the case for a participant who worked at a pre-school and appeared to 
provide extra services and niDre individual attention to the students. 
However, the school did not serve -^ny additional students because of the 
participant, and it did not receive any extra revenue. The alternative 
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supplier's price was just over the minimum wage, but the net value was zero 
because no increase occurred in the aggregate value of output* Thus, in 
some cases, the net value fails to capture quality changes* 

The net supply price is the appropriate concept when using the 
resource cost approach* Nevertheless, the gross supply price is useful for 
assessing the potential of participants to be productive* Table C*2 
presents both estimates* 

The table also presents the average participant compensation per 
active month and the average service project revenue per active month* 
These figures, which correspond to the steady-state period, provide an 
interesting basis for interpreting the value-of-output estimates, 
particularly when all value-of-output estimates are made on a per- 
participant basis* As shown in Table C*2, average revenue was only 35 
percent of the average net value added* However, average participant 
compensation was only slightly less than oi>.- estimate of net value added* 

In assessing how assumption about the value of Phase 1 and Phase 2 
output affect the overall beneft-cost results, we used the average revenue 
estimate as a lower bound* We used the net value added for the research 
sample as our preferred estimate* The average alternative supplier's price 
for the research sample provides a likely upper bound* Section F examines 
how these three alternative values affect the overall results* 

D. IMPACT ESTIMATES FOR THE BENEFIT-COST ANALYSIS 

The impact estimates used in the benefit-cost analysis are 
essentially those that were presented in Chapters IV through VII, with 
three differences* First, as we stated in Chapter VlII, we do not use the 
three separate point-in-time estimates of impacts at 6, 15, and 22 months 
after randomization* Instead, we linearly interpolate between those point- 
in-time estimates in order to estimate the cumulative impact over the 
entire 22-month observation period* Second, for dollar-denominated impacts 
(earnings, income, SSI/SSDI payments, and other welfare payments), we 
measured impacts in 1982 dollars rather than in current dollars, which were 
used in the impact analysis (this procedure was described in Section A)* 
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TABLE C.2 



A1.TERNATIVE ESTIMATES OF THE VALUE OF OUTPUT PRODUCED OH 
PHASE 1 AND PHASE 2 JOBS 



Valuation 
Method 



Value Per Active Month 



Net Value Added 

Full sample 
Research sample 

Alternative Supplier's 
Price'^ 

Full sample. 
Research sample 

Steady-State Phase 
Revenue^ 

Steady-State Participant 
Compensation 



Average Value . 

Phase 1 Phase 2 Both Phases Per Participant' 



281 
293 



326 
346 



n.a. 



n.a. 



498 
503 



681 
707 



n.a. 



n.a. 



n.a. 
n.a. 



n.a. 
n.a. 



131 
341 



3,446 
3,531 



4,391 
4,578 



1,223 
3,176 



The per-participant estimates are made by multiplying the estimates of the 
value of output per active month by the average length of participation. On 
average, participants spent 5.51 months active in Phase 1 and 3.81 months 
active in Phase 2, for a total length of stay of 9.32 months. These estimates 
are not discounted. 

b^^ 

"Alternative supplier's price" is the alternative labor cost of producing the 
output produced by the participant. "Net value added" subtracts out any 
additional supervision or lost productivity costs incurred by employers who 
provide Phase 1 and Phase 2 jobs. These values-of-output estimates are based 
on a randomly selected sample of 40 participants; 33 of these selected 
participants were in the STETS research sample. 

c 

Project revenue was received with a lag. The figure in the table assumes that 
the lag was three months after the work was performed (Riccio and Price, 
1984). If no lag was assumed, the project revenue per active month would have 
been $106 during the steady-state period. 

n.a. « not available. 
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Third, we disaggregated the living arrangement, schooling, aud training 
variables differently than was done in the impact analysis, so as to enable 
us to examine changes in the use of specific types of these programs and to 
value those changes separately. 

Table C.3 presents the point-in-time and full-period impact 
estimates. In most cases, the point -in-tiaje estimates were developed by 
using ordinary least squares regression methods. The exceptions are the 
two schooling variables and the four living arrangement variables, for 
which estimates are the difference between the means for the experimental 
and control groups. Both types of estimation procedures yield unbiased 
estimates of the impacts of STETS. In all cases, the impacts presented 
here are consistent with those presented in Chapters IV through VII.i 

The following are the specific sources of the impact estimates 
presented in Table C.3: 

• Non-STETS earnings: regression estimates using 
inflation-adjusted data and excluding all STETS 
earnings (estimates without these adjustments are 
presented in Table IV. 4) 

• Income: regression estimates using inflation-adjusted 
data (unadjusted estimates are presented in Table 
VII,5) 

• Sheltered workshop use: regression estimates are taken 
from Table IV. 4 

• Schooling variables (2 variables): exporimental- 
control difference in means (Table V.5 presents 
regression-based estimates of impacts on the other 
school variables) 

• Non-STETS training: regression estimates using data 
that excluded STETS training (Table V.5 presents 
regression-based estimates, including all training) 

• Living arrangement variables (4 variables): 
experimental -control differences in means (Table VII. 7 
presents similar estimates) 

• Receipt of transfers (3 variables): regression 
estimates (taken from Table VI. 3) 
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TABLE C.3 

WAO ESHHATES FCM THE CcNEFIT-COST AMALYSIS 





Units 


Point-In-Tias EstiMtss 




Period Esti«ates 




6 Months 


15 Months 


22 Months 


Months 1-6 


Months 7-15 


Months 16^22 


Month 22* 


Nmi*STET$ Earnings 


$fwk 


-11 .a 


i.97 


9.20 


-151.3 


-H9.6 


199.7 


39.87 


IneoM 


$/mk 


20.11 




BM 


261.%1 


526.%6 


233.26 


36.79 


Honths oft 


















Slitlttrid wortahop 




-0.0«3 


-0.0>7 


-0.113 


-0.189 


-0.720 


-0.735 


-0.113 


Nfular Moondary •diool 




-0.047 


-0.026 


-0.0%3 


-0.201 


-0.%19 


-0.2%2 


-0.0%3 


Otiwr MeomUry tehool 




-0.030 


-0.017 


0.006 


-o.;i% 


-0.2M 


-0.039 


0.006 


Nofi-SIETS training prograa 




-0.330 


-0.135 


-0.129 


-0.990. 


-2.093 


-0.92% 


-0.129 


Canttr/intt itut ion 




0.000 


-0.006 


-0.011 


0.000 


-0.027 


-0.060 


-0.011 


&oup horn 




-0.012 


0.009 


-0.012 


-0.036 


-0.01% 


-0«01) 


-0.012 


Fotttr horn 




0.00% 


-O.OOS 


0.006 


0.012 


-0.018 


-0.007 


0.006 


SMi*intf«ptndMit living prograa 




-0.012 


0.021 


0.022 


-0.036 


0.0%1 


0.151 


0.022 


SSI/S9DI 




-0.0%7 


-0.076 


-0.053 


-O.HI 


-0.55% 


-0.%S2 


-0.053 


Othav ^Ifar* 




-0.063 


-0.023 


0.0%6 


-0.249 


-0.%77 


0.081 


0.0%6 


N«ticaid/H«diear« 




-0.062 


-0.05% 


-0.022 


-0.186 


-0.522 


-0.266 


-0.022 


Trans f«r Pay«»nts froat 


















S9/$S)X 


$/ao 


-10.%3 


-17.53 


-17.31 


-31.29 


•125.82 


•121.9% 


•17.31 


Otfisr «slfar« 


$/ao 


-7.55 


-7.19 


9,%1 


-23.85 


•68.13 


7.77 


9.%1 



a 

Thasa aatijaataa raflact tha aatiMtad li^^cu for tha taanty-saeond aonth afUr raniloaizatlon. Thasa aati«atas font tha basis for tha 
axtr elation of tha aatlMtad impact s. 
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• Transfer payments (2 variables): regression estimates 
using inflation-adjusted data (Table VI, 3 presents 
estimates without this adjustment) 



Table C#4 illustrates the process used to estimate cumulative 
impacts for the out-of-program output variable* The top panel of the table 
indicates how the estimate of total compensation (gross wages plus fringe 
benefits) was derived from the interview data, which included only net 
(i.e., after-tax) wages. This process is described in the next section 
(Section E.l). The bottom panel of Table C.4 summarizes the interpolation 
process. The first step was to average the point-in-time estimates for the 
three periods— months 1 to 6, months 7 to 15, and months 16 to 11. These 
averages were then multiplied by the length of the associated periods to 
estimate the cumulative impacts. For example, consider the seven-month 
period from month 15 to month 22. The two point-in-tlme estimates are 
$5.76 per week (the impact at month 15) and $13.36 per week (the impact at 
month 22). Their average is $9.56 per week. This estimate is then 
multiplied by 30.3 (the number of weeks in the seven-month period) to yield 
an estimate of the average cumulative impact over that period, $290 per 
participant. This figure is then discounted to the time of enrollment 
using a real annual discount rate of 5 percent. The resulting present 
value for that period is $268 per participant. This procedure was used for 
all impacts in all time periods to estimate the cumulative impacts 
presented in Table C.3. 



E. ESTIMATED VALUES FOR IMPACTS 

The impact estimates presented in Table C.3 form the basis of the 
benefit-cost analysis. However, these estimates are generally not 
denominated in dollars and must be valued as part of the benefit-cost 
analysis. This section presents the procedures for valuing these impacts. 

1. Non-STETS Output 

The benefit-cost analysis incorporates this impact by valuing the 
increase in the output produced by participants. This value is estimated 
by using the earnings-impact estimate as the basis for estimating the 
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TABLE C.4 



ESTIMATES OF CHANCSS IN OUT-OF-PROGRAM OUTPUT 
(1982 dollars) 









Point-In- 


-Time Estimates 


Desc ription 


Units 


Baseline 


Month 6 


Month 15 


Month 22 


Change in Out-of -Program 
After-Tax Earnings 


$/week 


0 


-11.64 


3.97 


9.20 


Change in Out-of-Program 
Pre-Tax Earnings^ 


$/week 


0 


-14.32 


4.88 


11.32 


Change in Total Ouc-pf- 
Program Compensation^ 


$/week 


0 


-16.90 


5.76 


13.36 



Cumulative Estimates 



Months Months Months 
Description Units 1-6 7-15 16-22 Total 



Cumulative Change in Out- 
of-Program Output^ 


$/participant 


-220 


-217 


290 


-147 


Discounted Value of 
Cumulative Changes in 
Out-of-Program Output^ 


$/participant 


-217 


-208 


268 


-156 


NOTE: Details do not always 


sum to totals 


because of 


rounding. 







Pre-tax earnings are estimated as 1.23 times after-tax earnings to reflect the 
withholding of 18.7 percent of pre-tax earnings, the rate for minimum-wage 
workers. 

b 

Non-wage componeu s of total compensation (the difference between pre-tax 
earnings and total compensation) are estimated to be 18 percent of pre-tax 
earnings. 

c 

The cumulative changes in total compensation are estimated by linearly 
interpolating between the point estimates. 

d 

A 5 percent real annual rate is used to discount the value of effects to the 
time of enrollment. 




impact on total compensation (i.e., wages plus fringe benefits). Total 
compensation will equal the value of output produced if markets function 
competitively. Specifically, if the firms are prof it-^ximizers and 
compete in competitive product markets, they will then continue to hire 
workers to a point at which the contribution made to total output by the 
last worker hired (i.e., the value of the marginal product of labor) is 
equal to the compensation rate of the workers. At the same time, if labor 
markets function competitively, workers can move freely between firms in an 
attempt to obtain higher wages, so that all firms will have to provide 
equal compensation rates to workers who exhibit the same skill level. 
Together, these actions will lead to an equality between the total cost of 
an employee (i.e., the worker's total compensation) and the value of goods 
and services produced by that worker. Thus, under the assumption that 
sample members work in markets that can be characterized ^s competitive, 
the value of their output in those markets can be assumed to equal the 
compensation they receive. 

The interview data pertained to after-tax earnings rather than to 
total compensation. Thus, we had to multiply the impact estimate by an 
estimate of the average tax-withholding rate and by an estimate of the 
average fringe-benefit rate, where both rates pertain to workers in jobs 
such as those held by sample members. 

The tax-withholding rate was estimated from Internal Revenue 
Service regulations. In 1982, the minimum withholding rate for federal 
income-tax purposes was 12 percent. At the same time, the FICA tax rate 
for Social Security was 6.7 percent. Thus, the combined rate (ignoring 
withholdings for state and local taxes) was 18.7 percent of gross wages, 
or, equivalently, 23 percent of after-tax wages. 



1 

Because sheltered workshops are required to pay compensation that 
reflects productivity, this argument can reasonably be assumed to apply to 
sample members in both sheltered workshop and regular jobs. The 
correspondence between, total compensation and the value of output produced 
is less clear for persons in subsidized training jobs. 
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The U.S. Department of Labor (1980) estimated that wages and 
salaries for private nonfarm employees in 1977 accounted for 84.6 percent 
of total compensation. The remaining 15.4 percent consisted of supplements 
to wages and salaries, which included employer expenditures for retirement 
programs (both private retirement and Social Security), life, accident, and 
health Insurance, Workers' Compensation, Unemployment Insurance, and 
savings and thrift plans. These percentage estimates imply that the 
fringe-benefit rate for these employees is equal to 18.2 percent of wages 
and salaries. 

Smeeding (1981), providing a more recent estimate of fringe-benefit 
rates, used microsimulation techniques to estimate the average cost of 
fringe benefits to employers in 1979 and disaggregated the estimate by the 
yearly earnings of the employees. His estimates suggest that. In 1979, 
workers ^o were earning $10,000 or less received fringe benefits worth 
approximately 17.9 percent of wages and salaries, as compared with 19.8 
percent for all workers. We have used an estimate of 18 percent, which 
lies between the Smeeding and DOL estimates. 

2. Use of Other Programs 

STETS-induced changes in the use of other programs were valued by 
multiplying the estimated impact on the average months of use by the 
average cost per person month of serving persons in those other programs. 
We examined four alternative programs: sheltered workshops, secondary- 
level vocational programs, other school programs, and other job-training 
programs. We also consivlered four types of residential programs: 
institutions or centers, group homes, foster care, and semi-independent 
living programs. For all these programs, we used published estimates of 
average costs; when published estimates were unavailable for 1982, we 
multiplied the available estimate by the associated change in the implicit 
price deflator for gross national product. Table C.5 summarizes the values 
and sources used in the analysis. 



1 

Those with the lowest earnings, $2,000 per year or less, received 
an estimated 15.9 percent, while those with earnings of $7,501 to $10,000 
received an estimated 20.4 percent. 
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TABLE C.5 



ESTIMATED AVERAGE MONTHLY COSTS FOR SCHOOL, 
TRAINING, AND RESIDENTIAL PROGRAMS 



Program 


Average Cost per Month 
(1982 dollars) 


Data Source for Estimate 


Sheltered Workshop^ 


493 


U.S. Department of Labor 
(1977) 


Secondary-Level Vocational 
Education^ 


520 


Kakalik et al. (1981) 
Table 2.5 


Other School Programs^ 


290 


Dearman and Plisko (1982) 
Kakalik et al. (1981) 
Grant and Eiden (1981) 


Other Job Training^ 


113 


Kakalik et al. (1981) 


Center /Institution 


2,060 


nduucL eu ax. \LyoH/ 
Table 23 


Group Home^ 


1,239 


Hauber et al. (1984) 
Table 23 


Foster Care 


490 


Hauber et al. (1984) 
Table 23 


Semi-Independent Living 
Program 


830 


Hauber et al. (1984) 
Table 23 



This excludes capital costs. The cost nets out the revenue received from 
sales of workshop-produced goods and services. 

b 

This is a weighted average for serving learning-disabled students and 
educable and traina'^le mentally retarded students. The weights are the 
proportions of borderline, mild, and moderately retarded sample members. 

c 

This value is the weighted average of the costs of college, $368/month 
(Dearman and Plisko, 1982), regular secondary education, $284/month (Kakalik 
et al., 1981); and postsecondary vocational education, $163/month (Grant and 
Eiden, 1981). The weights are the proportions of sample members in these 
types of school at 22 months after randomization. 

d 

This value is for a work-study program. The other common type of job- 
training program for STETS sample members is on-the-job (OJT) training 
programs. Taggart (1981) estimates that OJT programs fund 2d by CETA cost 
$121/month (in 1982 dollars). 

e 

The group home estimate is a weighted average of the average cost of small 
group homes (fewer than 16 residents) and larger group homes (16 or more 
residents). The weights are the proportion of sample members in each 
facility type. 
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3» T ransfer Programs 

Changes in the use of SSI/JSDI, other welfare programs, and 
Medlcald/Medlcare enter the benef lt**co8t analysis in two ways* Reductions 
in the amount of transfer payments will be viewed as a cost by participants 
and as a benefit by taxpayers* The value of these transfers will cancel 
out from the social perspective and, hence, will not enter the social 
benefit-cost calculation* Reductions in program administrative costs — the 
amount of resources devoted to making the transfers —will appear as a 
benefit to taxpayers* Because an actual resource savings occurs, the 
administrative cost savings will enter the social calculation* 

We estimated the value of changes in payments from SSI/SSDI and 
other welfare directly from interview data, after adjusting for 
inflatio; We estimated the average monthly administrative cost for 
SSI/SSDI, $15 in 1982 dollars, by using information in the Budget of the 
United States Government (1982)* The monthly administrative costs for 
other welfare programs were assumed to equal the administrative costs per 
recipient for the Aid to Families with Dependent Children (AFDC) program* 
This cost was approximately $13 per month in 1982 dollars (Budget of the 
United States Government, 1982)* 

For Medicaid/Medicare, the average administrative cost per month of 
eligibility was estimated as $13, the average monthly Medicaid 
administrative expenditure per recipient (Budget of the United States 
Government, 1982)* The value of Medicaid coverage was estimated as $250, 
the average Medicaid expenditure per month (in 1982 dollars) for persons 
who are disabled (Sawyer et al*, 1983, Tables 4*4 and 4*15)* 

The use of average costs to value the change in administrative 
costs probably overstates short-run effects* In the short-run, the changes 
in costs should be estimated by marginal rather than by average costs* 
Marginal costs (the change in total costs due to a change of one recipient) 
are probably quite low for transfer programs, becouse staffing and 



1 

If all Medicaid recipients were included, the average monthly 
expenditure would be only $105* 
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facilities decisions are probably unaffected by small changes in case- 
loads. However, in the long-run, these decisions are more easily affected 
by caseload changes, so that average costs may better approximate long-run 
marginal costs. 

4. Tax Payments 

Participants' incomes will change as their earnings and transfer 
payment receipts change. These changes will cause changes in tax payments, 
including changes in income, payroll, sales, and excise taxes. Changes in 
the payments of these taxes were estimated by multiplying the average 
estimated change in participants' incomes by an estimate of the tax rate 
applicable to total income for low-income households, as obtained by 
Pechman (1985). His estimates suggest that persons in low-income 
households (i.e., households in the bottom 20 percent of the income 
distribution) face an average tax rate of 23 percent.^ 

In interpreting this rate, we should note that for households at 
the low end of the income distribution a major form of taxation is sales 
and excise taxes. Because these taxes are based on consumption and sales 
collected by retail firms, it is very difficult to avoid paying them.^ The 
other major form of taxation for these households is the payroll tax, 
which, when combined with sales and excise taxes, accounts for 80 percent 
of the total tax burden of low-income households. Thus, even though these 
households face low rates for the individual income tax, their total tax 
burden as a percentage of income is not subtantially different from the tax 
burden of most taxpayers. It should also be noted that Pechman's estimates 
correct for the general level of income underreporting for tax purposes. 
He also estimates the incidence of taxes for which the statutory and 



1 

Pechman presents tax rates for eight sets of incidence 
assumptions. In determining the 23 percent rate used here, we dropped the 
alternatives that created the highest (28.9 percent) and lowest (20.6 
percent) tax rates and averaged the remaining six. 
2 

However, it could be argued that the incidence is at: least 
partially incurred by producers. 
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ultimate tax burdens differ substantially (e.g., property and corporate 
income taxes> which may be shifted to workers and consumers). 

Once the rate was estimated, we calculated total income. We 
defined income in the same manner as in Chapter VII, with the exception 
that we added estimated tax withholding and fringe benefits to 
participants' after-tax earnings when estimating the earnings component of 
total income. 

?. SENSITIVITY TESTS 

It is evident from this discussion that numerous assumptions are 
involved in estimating net present value. Thus, a critical component of 
the benefit-cost analysis is to assess the importance of these various 
assumptions. We do so by examining the effect on the basic net present 
value estimate of changing sets of underlying assumptions while keeping all 
other assumptions unchanged. The pattern of estimates that are derived 
from these sensitivity tests indicates the overall level of uncertainty in 
the results. If the estimates change substantially and suggest different 
quali tative conclusions, then little confidence can be placed in the 
results. Alternatively, if the qualitative conclusiomi remain the same 
under a plausible range of alternative assumptions, then more faith in the 
conclusions is warranted. In addition, the sensitivity tests highlight 
those underlying assumptions that are particularly important for the 
results. Changing some assumptions may cause virtually no change in 
estimated net present value, while changing others may cause substantial 
changes. The sensitivity tests can provide a sense of which assumptions 
and impact estimates are particularly important — v sense that is useful in 
interpreting the overall findings. 

Table C.6 presents alternative estimates of net present value under 
a variety of different assumptions about the value of impacts and costs. 



1 

Another concern is the precision with which STETS impacts are 
estimated. In this regard, it should be noted that the earnings effects 
and the effects on the use of alternative programs (the major benefits) are 
statistically significant, and the estimates have relatively small standard 
errors . Moreover, all estimates used are unbiased measures of program 
impacts. Thus, we feel confident that the impact estimates used here 
accurately reflect the effects of STETS. 
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TABLE C.6 



ALTERNATIVE ESTIMATES OF NET PRESENT VALUE PER PARTICIPANT 

(1982 dollars) 



a 



Estimated Net Present Value 



Valuation Assumptions Social Participant Nonpartlclpant 

Basic Assumptions** $-1,038 $2,111 $-3,149 

Extrapolation Beyond 22-Month 
Observation Period 

7 months; no decay of impacts 4 2,333 -2,329 

30 months; no decay of impacts 3,221 3,018 203 

10 years; 14 percent annual decay 5^237 3,447 1,791 

Value of Phase 1 and Phase 2 Output 

Value » revenue -3,280 2,111 -5,391 

Value « alternative supplier's -22 2,111 -2,133 
price 

Operational Cost Equals the Cost -3,552 1,555 -5,107 
Observed for the Entire 
Demonstration 

Discount Rate 

3 percent -1,025 2,131 -3,157 

10 percent -1,068 2,060 -3,128 

Indirect Labor Market -2,677 2,111 -4,788 
Effects 



A full description of each set of evaluation assumptions is provided in the 
text. 

b 

See Table VIII, 5 for details on the value of specific benefits and costs under 
the basic assumptions. 
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The specific estimated values change substantially, but the set of 
estimates seems to indicate a consistent conclusion — that STETS will be an 
economically efficient investment if the impacts that are observed 22 
months after randomization persist for as little as seven months, and if 
program operating costs can be held at the levels observed during the 
demonstration steady-state period. 

The assumptions used in the basic estimate are those that we 
emphasized in Chapter VIII and in this appendix: that no impacts occur 
beyond the 22-month observation period; that the value of Phase 1 and Phase 
2 output is the net value added observed in the case studies; that research 
costs and the extraordinary operating costs did not affect program impacts; 
that the real (i.e., net of inflation) discount rate is 5 percent per year; 
that no indirect labor-market effects occur; and that the interpolations 
between observed impacts are accurate (in addition, a number of other 
assumptions are less central to the analysis). Under these assumptions, we 
estimate that social benefits are approximately $1,000 per participant less 
than social copts. Substantial benefits occur for workshops and other 
alternative programs. Nonetheless, these benefits are not large enough to 
offset the program costs under the basic assumptions. 

This basic result almost certainly underestimates the true net 
present value, because at least some impacts should persist beyond the 22- 
month observation period. In particular, the impacts on earnings and 
alternative program use should continue for awhile. These impacts are 
relatively large and statistically significant. Furthermore, these impacts 
appear to be relatively stable between months 15 and 22. Thus, the basic 
net present value estimate probably undercounts the benefits associated 
with these impacts. 

While it seems clear that these benefits will persist, it is 
impossible to determine how long they will continue or whether the impacts 
will increase or decrease over time. It may be that the STETS model, which 
provides transitional rather than long-run services, generates impacts that 
decline shortly after the end of program services. Alternatively, some 
participants may have successfully made the transition into the competitive 
labor market and will continue to earn more than they would have in the 
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absence of STETS. In the absence of Information on long-run effects, we 
excluded all future impacts •from the basic benefit-cost estimate and 
examined potential future impacts in sensitivity tests. 

Table C.6 presents three sensitivity tests pertaining to impacts 
beyond the observation period. These alternative estimates, along with the 
basic estimates, suggest the plausible range of benefits. If the impacts 
that are observed at 22 months after randomization persisted at that level, 
social net present value would be positive in seven months, and 
nonparticipants would perceive a net benefit within 2.5 years. If we 
adopted a ten-year planning horizon and assumed that all impacts declined 
by one-half every five years (a 14 percent annual decay rate), then we 
would find that STETS would generate substantial net benefits from all 
three perspectives. All of these alternative assumptions are arbitrary. 
Nonetheless, they suggest that, as long as the impacts do not decay very 
quickly (and we have no evidence to suggest that they will), then 3TETS is 
likely to generate benefits that exceed costs from the persper^tive of 
society and participants. The issue is less clear for nonparticipants— 
STETS will need to produce relatively long-run impacts in order to generate 
a net economic benefit to nonparticipants. 

The estimates in Table C.6 also show that the assumptions made 
about the value of Phase 1 and Phase 2 output have .a large affect on the 
overall results. The social and nonparticipant net present value estimates 
change by over $3,200, depending on whether this output is assigned the 
lower-bound estimate (service project revenue) or a likely upper-bound 
estimate (gross alternative supplier's price). However, this range 
probably overstates the uncertainty surrounding the value of this output. 
As stated earlier, revenue almost certainly understates the actual value of 
the output because the demonstration projects focused on generating jobs 
rather than revenues. Moreover, the alternative supplier's price is 
probably too high because there were several instances in which participant 
labor did not lead to net increases in output. What these estimates do 
indicate is the potentially important role of Phase 1 and Phase 2 output, 
clearly, placing participants in productive jobs as quickly as possible can 
increase net benefits. 
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The next sensitivity test in Table C.6 shows the affect of 
including the exfa operational costs incurred because STETS was a 
demonstration. This estimate excludes estimated research costs , but 
includes all other project-level costs for operations and participant 
compensation. It indicates that STETS would be much less likely to be 
economically efficient if costs reached this level (almost $11,000 per 
participant rather than $9,400 estimated for the steady state). It is 
unlikely that all the extra costs associated with start-up and the 
relatively small scale of operations affected impacts. However, some 
uncertain^r• surrounds the actual costs. It will be crucial to monitor 
costs closely in any future STETS-type efforts and to try to keep them at 
the steady-state levels. This situation seems plausible given that (1) 
projects were able to operate at these cost levels for five months and that 
(2) the impacts estimated for persons who were served during the steady- 
state period seem to be larger than those estimated for participants who 
were enrolled and received the bulk of their services at other times (see 
Table IV. 7). Therefore, we feel that steady-state costs are the most 
appropriate costs for judging the economic efficiency of STETS. 

The discount-rate sensitivity tests show that the choice of a 
discount rate is not crucial to this evaluation. Increasing the discount 
rate to 10 percent a year reduces net present value by 2.9 percent ($30). 
Lowering it increases net present value by 1.3 percent. These changes are 
trivial within the scope of the evaluation. 

The last sensitivity test in Table C.6 indicates the effects of 
assuming that half of the work performed by participants would have been 
performed by other workers in the absence of STETS. It also assumes that 
half of the work foregone by participants when they entered STETS is 
performed by other workers. These alternative assumptions, which are 
discussed in Chapter VIII, affect only those benefits that are associated 
with participant output. Thus, even though they would substantially reduce 
the value of net present value during the observation period, it is still 
plausible that STETS would generate sufficient savings from the reduced use 
of other programs to pay for itself from the social perspective. 
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other MDRC Studies 
on the STETS Demonstration 



Supported Work for the Mentally Rgtarded: Launching the 

STETS Demonstration , Bangser, Michael; and Price, Marilyn. 1982. 

A Transitional Employment Strategy for the Mentally Retarded; 
The Final STETS Implementation Report . Riccio, Jcjnes; with 
Price, Marilyn. 1984. 

Lessons on Transitional Enployment: The STETS Demonstration for 
Mentally Retarded Workers . Bangser, Michael. (A monograph 
summarizing the research findings auid operational experience.) 



1985. 
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